Learning Dynamic Representations for Discourse Dependency Parsing
Authors: Tianyi Liu
Yansong Feng
Dongyan Zhao
Venue: EMNLP 2023 Findings (2023)
Keywords: Discourse dependency parsing
Transition systems
Dynamic sub-tree representations
Graph attention networks
TL;DR: We employ graph models to derive dynamic representations for transition states based on the sub-tree structures we obtain from the previous steps.

Abstract: Transition systems have been widely used for the discourse dependency parsing task. Existing works often characterize transition states by examining a certain number of elementary discourse units (EDUs), while neglecting the arcs  obtained from the transition history. In this paper, we propose to employ GAT-based encoder to learn dynamic representations for sub-trees constructed in previous transition steps. By incorporating these representations, our model is able to retain accessibility to all parsed EDUs through the obtained arcs, thus better utilizing the structural information of the document, particularly when handling lengthy text spans with complex structures. For the discourse relation recognition task, we employ edge-featured GATs to derive better representations for EDU pairs. Experimental results show that our model can achieve state-of-the-art performance on widely adopted datasets including RST-DT, SciDTB and CDTB. Our code is available at $\href{https://github.com/lty-lty/Discourse-Dependency-Parsing}{https://github.com/lty-lty/Discourse-Dependency-Parsing}$.

Learning Dynamic Representations for Discourse Dependency Parsing
Tianyi Liu and Yansong Feng'* and Dongyan Zhao'#
Wangxuan Institute of Computer Technology, Peking University, China Center for Data Science, Peking University, China fliu-tian-yi, fengyansong, zhaody jepku. edu.
Abstract
Transition systems have been widely used for the discourse dependency parsing task. Exist- ing works often characterize transition states by examining certain number of elementary dis- course units (EDUs), while neglecting the ares obtained from the transition history. In this paper, we propose to employ GAT-based en- coder to learn dynamic representations for sub- trees constructed in previous transition steps. By incorporating these representations, our model is able retain accessibility to all parsed EDUs through the obtained arcs, thus better utilizing the structural information ment, particularly when handling lengthy text spans with complex structures. For the dis- course relation recognition task, we employ edge-featured GATs to derive hetter represen- tations for EDU pairs. Experimental results show that our model can achieve state-of-the- art performance on widely adopted datasets including RST-DT, SciDTB and CDTB. Our code is available at https://github.comm coom/tuubboo/tt--- Ity/Discuurse LtyDDiscouree-Deeeddenny
Intra-sentence parsing
Inter-sentence parsing
docu-
investigate monolingual NMT
EDU 13: By pairing monolingual training data with an automatic backtranslation,
EDU 14 treat additional parallel training data,
Figure I: The hierarchical parsing process for structure prediction. Sub-trees are built for each sentence and subsequently merged into complete dependency for the entire document.
1 Introduction
Discourse parsing is the task to study the inner structure of documents by analysing the relation- ship between text spans known as elementary dis- course units (EDUs). It is important research topic in natural language processing and benefits many downstream tasks, including sentiment anal- ysis (Bhatia al., 2015), text categorization (Ji and Smith, 2017), summarization (Xu al., 2020) and so on.
satisfactory when compared with syntactic pars- ing, mainly altributed to the difficulty of encoding EDUs properly and transferring information over long text spans.
Utilizing hierarchical models is a natural way to alleviate the problem. Figure shows the hierarchi- cal parsing process of document with sentences, total of 19 EDUs. If parse the document in normal manner, it challenging connect EDU 4 with EDU 18 correctly due to their considerable distance. But it is much easier when we parse the document at intra-sentena level and inter sentene level, respectively.
Existing methods for discourse parsing include transition-based models (Jia et al., 2018; Yu et al.. 2018; Hung et al., 2020) and graph-based models (Li al., 2014 2a) Recent works also employ top- down models (Koto al., 2021; Zhang al., 2020; Zhang et al., 2021b). However, their results are not
Coorespoondigg author.

---

Recent research has demonstrated the advan- tages of hierarchical discourse parsers across transition-based, graph-based, and top-down mod- els. Kobayashi et al. (2020) utilizes a top-down parsing approach with three levels of granularity: paragraphs in each document, sentences in each paragraph, and EDUs in each sentence. Zhang et al. (2021a) adopts Eisner's algorithm (Eisner, 1996) for structure and relation prediction intra- sentence, intra-parte para-apr and intra-document lev- els. Zhou and Feng (2022) proposes hierarchical transition model to predict the dependency tree structure at intra-sentence and inter-sentence lev- els and employs a stacked LSTM (Hochreiter and Schmidhuber, 1997) model to predict the relations.
root EDU of the sentence. Furthermore, we im- prove the performance of discourse relation recog- nition by employing edge-featured graph attention networks (EGATs; Chen and Chen, 2021). When compared to existing sequence labeling models, EGAT's outperform in leveraging precise structure information and predicting relations between dis- tant EDU pairs. Experimental results demonstrate that by utilizing more contextual information with GATs, our models achieve state-of-the-ar the perfor- mance on RST-DT (Carlson et al., 2001), SciDTB (Yang and Li, 2018) and CDTB (Li et al., 2014b) for both structure prediction and relation recogni- tion.
Our main contributions are as follows: (1) We propose a new transition-based model that incor- porates GAT-based sub-tres encoder, effectively leveraging the parsing history. (2) We evaluate two different types of graphs and demonstrate that besides EDUs, the structure information also mat- ters in predicting head-dependant relations, espe- cially when dealing with long text spans with com- plex structures. (3) We apply EGATs, instead of sequence labeling models, to discourse relation recognition to better leverage structure information. (4) Our model outperforms existing methods by large margin in popular discourse parsing datasets of different languages and genres.
The previous transition-based and top-down models, despite benefiting from hierarchical struc- tures, suffer from common issue: they do not effectively utilize lower-level structures for upper- level parsing. Kobayashi et al. (2020) represents span as a whole by employing weighted sum over word representations. Zhang et al. (2021a) only assigns complete and incomplete scores to each span. Zhou and Feng (2022) focuses exclusively on root EDUs of sentences for inter-sentence pars- ing. These approaches fail to leverage the structural information acquired from preceding lower-level parsing stages during the upper-level parsing pro- cess. This may hurt the model performance. For example, during inter-sentence parsing in Figure 1, the key information in EDU 13 (monolingual, backtranslation) is crucial when we consider the relationship between EDU and EDU 14. There- fore, it is important for us to leverage the parsing history in subsequent steps.
Related work
Pre-trained Language Models Recent works have shown that pre-trained language mod- els(PLMs) have achieved good results in discourse parsing (Koto et al., 2021; Nguyen et al., 2021; Hung et al., 2020). These works prove that pre- trained language models can better capture rela- tionship between EDUs than hand-crafted features. Yu et al. (2022) points out that PLMs are trained with sentence-level contexts, rather than the EDU- level structure in discourse parsing. They per- form second-stage EDU-level pre-training with tasks, next EDU prediction and discourse marker prediction, and these adaptations achieve great im- provements. For fair comparison, our model and the baselines in this paper use the same PLMs: bert- base-uncased for English and bert-base-chinese for Chinese.
To address this issue, we propose solution in the form of a hierarchical transition-based model. Our approach mitigates the aforementioned prob- lem by employing graph attention networks (GATs Veličković al., 2018) to extract features for the sub-trees constructed the preceding parsing steps. In contrast to Zhou and Feng (2022), where an EDU becomes inaccessible once it is popped from the stack, our method maintains accessibility to it through the obtained ares, enabling its reuse in subsequent parsing procedures. For example, when we consider the relationship between EDU 4 and EDU 8, we can use information from both intra- sentence parsing history (arc >> 2) and parsing history of the current inter-sentence sequence (are 4 + 0), instead of relying solely on the isolated
Discourse Dependency Parsing Compared discourse constituency parsing, discourse depen- dency parsing receives relatively less attention. Li et al. (2014a) converts constituency trees in RST-

---

DT to dependency trees, and employ two graph models on the new dataset. Yang and Li (2018) presents SciDTB, a discourse dependency tree- bank for scientific abstracts from ACL Anthology and implements transition-based and graph-based parsers on the dataset. Yi et al. (2021) adapts three Chinese discourse corpora to the dependency framework. We use these datasets to evaluate our model.
process follows a bottom-up approach, where the sub-trees constructed in the preceding steps are utilized in the subsequent steps.
Figure I illustrates the parsing process employed for structure prediction. The document selected in SciDTB comprises five sentences with a total of 19 EDUs. Our approach involves initially constructing a sub-tree for each sentence, followed by merging these trbet to form a comprehensive tree repre- sentation.
Hierarchical Discourse Parsing Hierarchical models are employed to improve the discourse de- pendency parsing performance for raasition-basee (Zhou and Feng, 2022), graph-based (Zhang et al.,, 2021a), and top-down (Kobayashi al., 2020) models. These methods improve parsing results but they do not make full use of the parsing his- tory, as is described in the introduction part. We store the predicted arcs and expand each EDU to a sub-tree and use GAT-based encoder to merge information from EDUs in the sub-tree.
We adopt the hierarchical setting for several rea-
sons:
(1) The hierarchical structure reduces the aver-
age length of the sequence.
(2) Our approach aligns with the target of form- ing a tree structure.
(3) Observations of datasets: We define EDU as an out-EDDU if its head is in another paragraph. In RST-DT, 89.5% of paragraphs have only one out- EDU (root), and 86.1 % of out-EDUs have another out-EDU as its head. These results are acceptable due to the relatively loose connections between paragraphs. The percentage is even higher for sen- tences. These observations encourage us to employ this simplified approach.
3 Methodology
Given document composed of sequence of EDUs E E2......... en), discourse depen- dency parser is expected to predict a tree struc- ture that represents the relationships between the EDUs. For each EDU 6; E F, excluding the sin- gle root EDU of the document, a head EDU h4 is selected from the set of EDUs E, and a relation T; is assigned from a pre-defined set of relations R to describe the relationship between the EDU pair (ci, h:). We adopt pipelined approach (Wang et al., 2017) for the task. We construct a depen- dency tree structure for each document and then leverage this tree structure predict the relations between EDUs.
3.1.2 Transition System
At each level of structure prediction, we utilize an arc-eager transition system (Nivre, 2003) as our parsing approach. We employ a queue Q initialized with the input sequence and an initially empty stack S to store the EDUs. During the parsing process, maintain set A to record the acquired ares derived from the transition history.
For each transition state (2, S,A), the discourse parser takes one of the following actions: Shift, Reduce. LArc and RightAre. The Shift ac- tion removes the first EDU in the queue and pushes it to the top of the stack S. The Reduce action pops the top EDU from the stack S. The LeftAre frrre action adds an are to the set A, where the head is the first EDU in the queue Q and the dependent is the top EDU in the stack S. After adding the arc, the parser performs the Reduce ac- tion. The Right Arc action adds an are to the set A, where the head is the top EDU in the stack S and the dependent is the first EDU in the queue Q. After adding the arc, the parser proceeds with the Shi ft action. The parsing process continues until the queue Q is empty and only a single root EDU for the input sequence remains in the stack S.
3.1 Structure Prediction
In the structure prediction step, we utilize hierar- chical variant of transition systems. This section begins by introducing our hierarchical setting and transition system, followed by detailed implemen- tations of our model.
3.1.1 Hierarchical Setting
Our method parses the document at three levels for RST-DT: EDU-to-sentence (E2S), sentence- to-paragraph (S2P), and paragraph-to-documen (P2D). For SciDTB and CDTB, in which paragraph structures are absent, we refer to the first two levels as intra-sentence and sentence The parsing

---

3.1.3 Our Model
State Representations For each tran- sition state (Q) we consider the top two EDUs in the stack and the first two EDUs the queue Q as source EDUs. We expand each source EDU into a sub-tree and calculate the sub- tree representation as we mentioned above. Finally, we obtain representations for the transition state by concatenating the representations of the four sub- trees. To predict the transition action, we employ fully-connected layer.
Transition
In this section, we introduce how our transition system handles the current state.
(1) Select four source EDUs available, includ- ing the top two EDUs from the stack and the first two EDUs from the queue.
(2) Construct sub-tree for each source EDU with the ares obtained in previous steps.
(3) Derive contextualized representations for cach EDU in the sub-trees.
(4) Calculate representations for each sub-tree based on inner EDUs using GAT's
3.1.4 Sub-Tree Construction
In this section, we present two distinct methods for expanding source EDU into sub-tree: the Star Graph and the Full Graph.
(5) Concatenate the sub-tree representations and employ fully-comnected layer for predicting the subsequent action.
We will also elucidate the implementation details of our model, specifically focusing on the methods for deriving representations for EDUs, sub-trees, and transition states..
4
0
3
EDU Representations To capture contextual rep- resentations of EDUs, we employ pre-trained BERT (Devlin al., 2019) model to encode each EDU within its corresponding context. For RST- DT, we consider the entire paragraph as the context for each EDU. For SciDTB and CDTB, we pro- vide BERT with the sentence containing the EDU, well the preceding and succeeding sentences if available, to capture the contextual information. Sentences are delimited by [SEP] tokens, while each sentence comprises EDUs separated by addi- tional special tokens [FSS] representing "edu-sep". For each EDU, we compute its contextual repre- sentation by taking the token-level average of the corresponding EDU span.
1
2
Full Graph for EDU Star Graph for EDU 4
Figure 2: Full-Graph and Star-Graph for the source EDU 4. The source EDU is depicted with circular bor- ders, whereas expanded EDUs are represented with square borders. The graphs are constructed based ares stored in the mnove-mentioned transition states after three steps. The are between EDU and EDU comes from the current inter-sentence sequence, while the oth- ers come from intra-sentence sequences.
Star-Graph Model single EDU may not con- tain all the information we need and EDUs linked to it in precious steps may also convey important information. Therefore, we take these EDUs into consideration. Given the source EDU e, and its cor- responding sub-tree T. we construct star graph where e is directly connected to all the other EDUs within T, illustrated in Figure 2. Subsequently, we calculate weighted average V(T) over the EDUs T to represent the sub-tree.
Sub-Tree Representations In our hierarchical model, important to highlight that the set A not only stores ares generated from preceding steps within the current EDU sequence but also captures lower-level parsing results, which should be uti- lized in upper-level parsing. For instance, in inter- sentence parsing, even though only the root EDUs from the intra-sentence sub-trees are used as input for the model, the sub-trees can be reconstructed using the intra-sentence ares contained within A.
V(T) = Σ ((e)V(e) =
(1)
Given a source EDU, expand into dis- tinct types of sub-trees: the Star Graph and the Full Graph. The methods to construct the sub- trees and calculate the sub-tree representations will be introduced in detail in section 3.1.4.
eET
Here V(e) is the representation of EDU e. The weights w--) are determined by the similarity be- tween each EDU and the source EDU e.- We sim-

---

ply take the inner product as the similarity score.
attention in our model. To merge information from multi-hop neighbors, we utilize multiple graph at- tention layers. This allows us to incorporate knowl- edge from distant nodes in the graph. We also apply residual connections and layer normalization tech- niques to further enhance the model performance. We extract the representations of the source EDU in the final layer the sub-tree representations.
cap(sim(( (c),, (ee)))
((e) Enpr exp(sim(((ee)) V(e.))) (2)
Full-Graph Model The star graph connects all EDUs in the sub-tree directly to the source EDU eg destructing the original structure of the sub-tree. We argue that the structure can also convey valu- able information aside from the EDUs. For the full graph, retain the complete structure of T. We view the sub-tree T an undirected graph, where each node corresponds to an EDU. We lever- age GATs (Veličković et al., 2018) to incorporate neighboring information into the source EDU 63- In each graph attention layer, we conduct self- attention on node representations to calculate aij. which quantifies the significance of ej to
3.2 Relation Recognition
After the structure prediction step, we generate dependency tree that includes nodes represent- ing EDUs and edges representing relations be- tween them. For the relation recognition step, we fine-tune two distinct BERT models for the nodes and edges within the tree structure. Subsequently we employ edge-featured GAT's (Chen and Chen, 2021) to derive structure-awe aware contextualized rep- resentations for EDU pairs.
exp(sig)
jij Σ (((e) exp(s;))
Given constructed discourse de- pendency tree, we employ the same method de- scribed in section 3.1.3 to obtain representations for each EDU. For each EDU pair connected by an edge (es, h4), we concatenate their respective rep- resentations and utilize a fully-connected layer predict their relation. We fine-tune the NodeBERT model with gold dependency tree structures in the training set.
NodeBERT
The attention weight is applicable only when ey C N (ei). Here (ee)) denotes the set of first-order neighbors of e,, which includes itself. The simi- larity score sij is defined
8ij LeakyReLU (TT[Wo:llW))
(4)
Here LeakyReLU is defined follows and we set
x 0.2. Leu)))eLU(s)) =
EdgeBERT Similar to Zhou and Feng (2022), for each edge (e,, hτ) in the dependency tree, we con- catenate the two EDUs in the order they appear in the original sequence. The concatenated sequence is then fed into BERT model. Subsequently, the embedding of the special token [C'LS] is employed to predict the relation r; using a fully-connected layer. We also fine-tune the EdgeBERT model with gold dependency tree structures in the training set.
Jxr .220 0 Ax <
Us oy are representations for ex, ey, and the matrix W denotes a linear projection applied to the nodes. W Rdou xd,, and a E 2dont are learnable parameters. din refers to the dimension of EDU representations and is the concatenation opera- tion. Then the representations for e; can be updated follows:
Node+Edge Model After the fine-tuning step, we utilize NodeBERT to encode each node and EdgeBERT to encode each edge. For each edge (e,, h.) in the dependency tree, We concatenate the representations of the edge with those of the corresponding nodes and predict the relation using fully-connected layer.
1/f ELU (EjezeNte)) aiy wiyWw;;
(5)
Here ELU is defined follows and we set
1.
FLU((z) f A
x .220 0
x <
EGAT Model We view each EDU as node in the graph and head-dependant pairs as edges in the graph. Besides the edge itself and its corresponding nodes, the other nodes and edges may also contain important information for predicting the relation
According to Vaswani al. (2017), multi-head at- tention enhances performance by simultaneously capturing information from different representa- tion subspaces. Therefore we employ multi-head

---

for the edge. Based on the representations for nodes and edges, we apply EGATs (Chen and Chen, 2021) to the graph. EGAT is similar to GAT introduced in section 3.1.4, except for the following rules:
RST-DT
SciDTB
CDTB
#doc (train)
312
743
1599
#ldoc (dev)
30
154
350
#doc (test)
38
152
374
(1) Edge features are employed to compute atten- tion scores between nodes. We adapt the similarity score (Eq. 4) between node i and as follows:
#rel (coarse)
19
17
18
#tel (finc)
111
26
#para/docu
10.03
#scnt/para
2.01
5.33
2.03
#cdu/sent
2.76
2,63
2.23
Sij - LeakyReLUU [W us Way W Wyyl
#token/edu 10.85
11.10
22.76
(6)
Here, R.ddow Wx, We R. denote linear projections for nodes and edges, respectively. The representations for nodes and edges are de- noted as v and u.
Table 1: Detailed information about the datasets, in-
cluding the dataset split, the number of coarse-grained and fine- ane-grained relation types and the average number of paragraphs in documents, sentences in paragraphs, EDUs in sentences and BERT tokens in EDUs.
(2) In addition to updating node representations, we construct dual graph to update edge representa- tions. Each node in the dual graph G" corresponds to an edge in the original graph G. Two nodes in G' are connected if and only if their corresponding edges in G share common node. Consequently, we can update node representations in G" (as well as edge representations the original graph G) using the same method as described in (1).
We employ the unlabeled attachment score (UAS) and labeled attachment score (LAS) as eval- uation metrics to assess the performance of our model. The UAS measures the percentage of EDUs in which the model correctly predicts the head, while the LAS measures the percentage of EDUs where the model correctly predicts both the head and the relation.
(3) Self-loops are added for both G and G¹. In the original graph G, each node is assigned self- loop that aggregates the features of all edges con- nected it by taking their average. In the dual graph G², each node possesses a self-loop with zero-valued features. More details can be found in Chen and Chen (2021).
4.2 Baselines
Since there are only few works in discourse de- pendency parsing, we only compare our model with the state-of-the-art transition-based and graph- based models. For fair comparison, the pre-trained language models used are pert-base-nncasee for English and bert-base-chinese for Chinese.
After applying multiple EGAT layers to update the representations of nodes and edges, charac- terize each EDU pair (e,, h2) by concatenating the EDUs and the edge con-
(1) Zhou22 Zhou and Feng (2022) proposes hierarchical transition-based model for structure prediction and uses sequence labeling model predict discourse relations.
representations of the necting them. Finally, we utilize fully connected layer to predict the relation.
(2) Zhang21 Zhang et al. (2021a) utilizes a hierarchical Eisner model with neural CRF autoen- coders to simultaneously predict heads and rela- tions. For fair comparison, consider the fully- supervised version of their approach.
4 Experiments
4.1 Dataset and Evaluation Metric
We evaluate our model on three datasets: RST- DT (Carlson et al., 2001), SciDTB (Yang and Li, 2018) and CDTB (Li et al., 2014b). RST-DT is a dataset that consists of Wall Street Journal articles. SciDTB is dataset comprising scientific abstracts from the ACL Anthology. CDTB is dataset con- taining Chinese newswire articles. To convert the constituency trees into dependency trees, follow the approach proposed by Li et al. (2014a) for RST- DT and the method introduced by Yi al. (2021) for CDTB. More detailed information about these datasets provided in Table I.
4.3 Hyper-Parameters
Here we present the value of several important hyper-paranee rrperpparameter::
(1) The dimension of token, EDU and sub-tree representations is set to 768. In Equation 4 and 6, we set d,,, to 768 and dout to 768/ H, where H represents the number of attention heads.
(2) For GATs, we explore different settings for the number of layers (L) chosen from {1, 2, 3, 5), the number attention heads (H) selected from

---

stage
H
bs
epochs
CAS
LAS
(tructure(E2S)
3
16
2e-5
Zhou22-rep
79.3
65.0
S(FictureS2P&&F2D)
6 8
2e-5
SciDB
Zhou22-run
78.08 + 1.06
relation
3 32
10
Our model
81.28 0.34
67.31 0.44
Zhou22-rep
82.2
64.8
Table 2: Hyper-parameters in different stages of our model. The cell with asterisk: Ir=1e-6 for BERT parameters and le-4 for the others
CDTB
Zhou22-run
81.76 + 1.18
62.63 0.87
Our model
84.36 + 0,05
67.61 0.36
Table 3:
Discourse dependency parsing results on SciDTB (with 26 fine-grained relations) and CDTB (with 18 relations). We run both Zhou22(2hou22-rum and model for six times, and we report the mean and standard deviation of the results for each model. Our re-implemented results of Zhou22 is lower than those reported in the paper, so also list the reported results hero(Zhhu22-rep).
11,3, 6, 6, 12), batch size (bs) from (8, 16, 32), as well as the learning rate (Ir) and the number of training epochs.
We list the values in different stages in Table 2.
4.4
Main Results
SciDTB (c)
RST-DT (c)
We report the results of our model with Full Graph in section 3.1.4 for structure prediction and EGATs in section 3.2 for relation recognition, and compare our model with the baselines mentioned above.
UAS
LAS
UAS
LAS
Zhang21 Our model
79.1
65.0
70.2
51.8
81.28
71.23
71.41
55.64
We conduct comparison between our model and Zhou22 on SciDTB (with 26 fine-grained re- lations) and CDTB (with 18 relations). The corre- sponding results are presented in Table 3. These results demonstrate that our model surpasses the most recent transition baased model for discourse dependency parsing. The increase in UAS (3.20% for SeiDTB and 2.60% for CDTB) highlights the advantages of our GAT-based sub-tree encoder, as our model incorporates GATS to capture sub-tree representations for transition states instead of solely considering isolated EDUs The improvement in LAS (4.89% for SciDTB and 4.98% for CDTB) in- dicates the superiority of our EGAT- based relation classifier over sequence labeling models. By com- paring the mean and standard deviation of UAS and LAS score, we can see that our model outper- forms Zhou22 by a large margin in both structure construction and relation recognition, even when compared with the higher reported results, indicat- ing the superiority of our GAT-based model. We will provide comprehensive comparison between model and Zhou22 in section 4.5.
Table 4: Discourse dependency parsing results on SciDTB (with 17 coarse- relations) and RST-DT (with 19 coarse-grained relations). We use the reported results in Zhang21. For our model, we take the average results of three runs.
4.5 Detailed Results
Since our model follows similar pipeline proce- dure Zhou22, present the results of each step and conduct thorough comparison on SciDTB.
Discourse Structure Prediction We partition the EDUs, excluding the root EDU of each document, into two subsets: Eistra and Einter- An EDU is categorized as part of Fintra if it belongs to the same sentence as its head EDU, and as part Exnter otherwise. We evaluate model performance on these two subsets.
We conduct experiments at intra-sentence and inter-sentenes levels and define the Intra score the percentage of EDUs in Emtra for which the head EDU correctly predicted. Inter __ and In- ter -g are similar scores for Einter based on pre- dicted and gold intra-sentence structure separately.
Besides, we compare our model with Zhang21 on RST-DT (with 19 coarse-grained relations) and SciDTB (with 17 coarse-grained relations). As shown in Table 4, our model exhibits significant improvements over the latest graph-based discourse dependency parsing model, even when applied to longer documents in the RST-DT dataset. This finding suggests that our model effectively handles distant EDU pairs within a lengthy context.
We evaluate three models using these scores, all of which are transition- models. The pri- mary difference among them lies in the represen- tations of sub-trees. Zhou22 exclusively utilizes the source EDU for representation, therefore we refer to this approach as the No-Graph model. In contrast, our two graph models expand the source EDU to encompass a sub-tree by incorporating the

---

No-Graph
$
Intra_g
Inter
LAS_g
(Star Graph)
88.85 0.70
Ours (Full Graph)
89.09 = 0.35
Seq-label
82.4
62.2
77.4
Ours (Node+Edge)
83.5
64,4
78.6
No-Graph
63.21 1.49
Ours (EGAT)
83.3
66.2
79.0
Inter_p
Ours (Star Graph)
65.36 1.28
Ours (Full Graph)
67.81 0.88
Table Relation recognition results on SciDTB with fine ane-grained relations and gold tree structure. We use the results reported in Zhou22 for the Seq-label model, and our model, we report average results six runs.
No-Graph
66.54 1.30
Inter
Ours (Star Graph)
68.73 1.13
Ours (Full Graph)
71.16 0.66
Table Structure prediction results on SciDTB. ob- tain results for the No-Graph model, we re-implement the approach described in Zhou22, for some of the scores defined here not reported in the original pa- per. We present here the mean and standard deviation results for six runs.
ter ter_g and LAS_ as the accuracy score for rela- tion recognition in Eintra. Einter and all the EDUs. Zhou22 treats discourse relation recognition as a sequence labeling task, and we refer to it as Seq- label. We compare it with our Node+Edge and EGAT models introduced in section 3.2. The sults presented in Table 6.
ares obtained during preceding parsing steps. Our Star-Graph model constructs star graph by di- rectly connecting all other EDUs in the sub-tree to the source EDU and our Full-Graph model per- forms GATs on the original sub-tree structure. We provide detailed explanations in section 3.1.4. The results are presented in Table By comparing the mean and standard deviation of the results, we can draw the following conclusions.
Edge achieves superior performance com- pared to Seq-label primarily due to the integra- tion of two fine-tuned BERT models. NodeBERT captures contextual representations for each EDU, while EdgeBERT facilitates direct interactions be- tween EDU pairs. The Seq-label model contains only the latter. EGAT outperforms Seq-label at inter-sentence level, further emphasizing the signif- icance of graph structures in handling complex spans.
For intra-sentence parsing, both our Star-Graph and Full-Graph models surpass the performance of the No-Graph model, emphasizing the need to integrate information from all EDUs within the sub-tree. However, the Full-Graph model demon- strates only marginal improvements compared to the Star-Graph model. This observation can be attributed to the relatively small number of EDUs in each sentence and the simple sentence structures, suggesting that simplified star graph is sufficient for merging information between EDUs within a sentence.
Analysis
It evident that discourse dependency parsing per- formance declines when applied to lengthy text spans with complex structures. In this section, we conduct experiments on SciDTB to show the capa- bility of our model in handling complex structures. Additionally, we perform experiments on RST-DT to showcase the superiority of our model in dealing with long text spans.
For inter-sentence parsing, our Star-Graph and Full-Graph models outperform the No-Graph model by larger margin. This superiority arises from their ability to leverage ares obtained from intra-sentence parsing to enhance inter-sentence parsing. Unlike results in intra-sentence parsing, the Full-Graph model significantly outperforms the Star-Graph model. This observation high- lights the critical role of the graph structure in comprehending text spans with larger number EDUs and more complex structures.
5.1 Results over complex structures
We perform inter-sentence experiments on SciDTB using gold intra-sentence structures. We divide the root EDUs sentences into subsets based on the number of EDUs in the sentence and the sentence containing the head EDU.
The results are presented in Table 7. For most of the subsets. both the Star-Graph and Full-Graph models outperform the No-Graph model, indicat- ing the necessity of merging information from all the elementary discourse units (EDUs) within a sub-tree. Furthermore, when comparing the Full- Graph model with the Star-Graph model, ev-
Discourse Relation Recognition We evaluate discourse relation recognition models based on gold dependency trees. We define Intra In-

---

N_edu
No-Graph
Star-Graph
Full-Graph
No-Graph
Star-Graph
Full-Graph
2 (24)
66.67
65.28
64.58 (-0.70)
P2D P P2D_g g
25.6
22.4
30.0
3 (98)
58.50
62.24
62.41 (+0.17)
36.3
33.3
41.1
4 (148)
65.99
64.98
66.55 (+0.57)
(157)
63.06
63.27
65.50 (+2.23)
Table 8: P2D-level structure prediction results on RST.
(108)
66.36
68.21
72.53 (+4.32)
DT.
(69)
58.94
59.66
64.01 (+4.35)
(30)
64.44
63.89
73.33 (+9.44)
28 (32)
58.31
58.84
62.50 (+3.66)
Diu_
No-Graph
Star-Graph
Full-Graph
Table 7: Inter-sentence level structure prediction re- sults in different subsets of SciDTB. The numbers in the brackets in the first column represent the size of each subset and the numbers in the brackets in last column indicate the performance gain Full-Graph achieves compared to Star-Graph.
1-4 (130)
28.5
50.0
5-8 (110)
31.0
28.2
41.0
9-18 (106)
22.6
20,8
35.9
>18 (93)
21.5
35.5
12.9
Table 9: P2D-level structure prediction results in differ- ent subsets of RST-DT. The numbers the brackets the first column represent the size of each subset.
ident that the Full-Graph model achieves greater improvements when there more EDUs in the corresponding sentences. When sentences contain a higher number of Elementary Discourse Units (EDUs), they often exhibit more complex struc- tures. This finding demonstrates the capability of our GAT-based sub-tree representations to handle text spans with complex structures.
two EDUs is measured by the number of EDUs be- tween them, with a value of I for adjacent EDUs. We compute the P2D _ scores for these subsets and present results in Table 9. Based on the obtained results, we can draw the following con- clusions: (1) Our Full-Graph model demonstrates superior performance compared to the No-Graph model across all subsets, except for the subset in- volving extremely long distances. Furthermore, this superiority becomes more pronounced the subsets involve longer distances. These findings indicate that our Full-Graph model exhibits a re- markable capability to handle longer spans. (2) The Star-Graph model yields a lower score for adjacent EDU pairs but excels in the case of distant ones. This observation provides evidence that the destruction of the sub-tree structure indeed results in data contamination mentioned above. The ef- fectiveness of our GAT-based sub-tree encoder is validated by these experiments.
5.2 Results over long text spans
We conduct experiments with our model on RST- DT, dataset consisting of documents with erage of 10.03 paragraphs. We evaluate the results of P2D-level parsing. For the score P2D g. use gold E2S and S2P structures and for P2D_p. we utilize E2S and S2P structures predicted by our Full-Graph model.
We present the performance of three approaches in Table 8: No-Graph, Star-Graph, and Full- modified
Graph. The No-Graph approach version of our model where the sub-tree represen- tation utilizes only the source EDU. Based on the results, it is evident that our Full-Graph model sur- passes the performance of the No-Graph model, highlighting the superiority of our GAT-based sub- tree encoder. The Star-Graph model achieves lower score, possibly due to the destruction of the original sub-tree structure. Since there are numer- ous EDUs in a sub-tree during Sev-level parsing, when we simplify the sub-tree to a star graph, the information carried by nearby nodes may be con- taminated by distant nodes.
Conclusion
In this paper, we enhance hierarchical transition systems by integrating a GAT-based sub-tree en- coder. which enables the parser to leverage the parsing history more effectively and have a better understanding of the whole document. Addition- ally, we apply edge-fcatured GATs to predict re- lations based on the complete discourse structure, where two separate BERT models are fine-tuned capture features for EDUs and EDU pairs, spectively. Experimental results demonstrate that our model outperforms existing methods, particu- larly when applied to long text spans with complex structures.
For further study on this topic, we partition the root EDUs of paragraphs into subsets of approx- imately equal size, taking into account their dis- tance from the head EDU. The distance between

---

Limitations
Shyh-Shiun Hung, Hen-Hsen Huang, and Hsin-Hsi Chen. 2020. A complete shift-reduce Chinese dis- course parser with robust dynamic oracle. In Pro- ceedings of the 58th Annual Meeting of the Associa- lion for Computational Linguistics, pages 133-138, Online. Association for Computational Linguistics.
Since the sub-tree changes during the transition- based parsing process, we have to encode different sub-trees for each transition state, which is time- consuming. As a result, our structure prediction model requires more time compared to existing models. This issue becomes particularly prominent when dealing with long documents in RST-DT. In future works, We will design a more efficient graph structure for encoding sub-trees. Besides, when evaluating our model on CDTB, we encounter a sig- nificant issue of data imbalance, particularly in the stage of inter-sentence structure prediction, where Shi,ft and RightAre comprise the majority of the actions. Consequently, our model has tendency to produce fully right-branching trees. We leave the problem for future works.
Yangfeng Ji and Noah A. Smith. 2017. Neural dis- course structure for text categorization. In Proceed- ings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 996-1005, Vancouver, Canada. Association for Computational Linguistics.
Yanyan Jia, Yuan Ye, Yansong Feng, Yuxuan Lai, Rui Yan, and Dongyan Zhao. 2018. Modeling discourse cohesion for discourse parsing via memory network. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 438-443, Melbourne, Australia. Association for Computational Linguisties.
Naoki Kobayashi, Tsutomu Hirao, Hidetaka Kamigaito, Manabu Okumura, and Masaaki Nagata. 2020. Top- down rst parsing utilizing granularity levels in doc- uments. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):8099 -006.
Acknowledgements
This work is supported by NSFC (62161160339). We would like thank the anonymous reviewers for their helpful comments and suggestions.
Fajri Koto, Jey Han Lau, and Timothy Baldwin. 2021. Top-down discourse parsing via sequence labelling. In Proceedings of the 16th Conference of the Euro- pean Chapter of the Association for Computational Linguistics: Main Volume, pages 715-726, Online. Association for Computational Linguistics.
References
Parminder Bhatia, Yangfeng Ji, and Jacob Eisenstein. 2015. Better document-level sentiment analysis from
RST discourse parsing. In Proceedings of the 2015 Conference on Empirical Methods Natural Lan- guage Processing, pages 2212-2218, Lisbon, Portu- gal. Association for Computational Linguistics.
Sujian Li, Liang Wang, Ziqiang Cao, and Wenjie Li. 2014a. Text-level discourse dependency parsing. In Proceedings of the 52nd Annual Meeting of the As- sociation for Computational Linguistics (Volume I: Long Papers), pages 25-35, Baltimore, Maryland. Association for Computational Linguistics.
Lynn Carlson, Daniel Marcu, and Mary Ellen Okurovsky. 2001. Building a discourse-tagged cor- pus the framework of Rhetorical Structure Theory. In Proceedings of the Second SIGdial Workshop on Discourse and Dialogue.
Yancui Li, Wenhe Feng, Jing Sun, Fang Kong, and Guodong Zhou. 2014b. Building Chinese discourse corpus with doniective-driven dependency tree struc- ture. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2105-2114, Doha, Qatar. Associa- tion for Computational Linguistics.
Jun Chen and Haopeng Chen. 2021. Edge-featured graph attention network. CORR, 2e//2101.076711
Jacob Devlin, Ming- Mei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume I (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Thanh- Tung Nguyen, Xuan-Phi Nguyen, Shafiq Joty, and Xiaoli Li. 2021. RST parsing from scratch. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 1613- 16653, Online. Association for Computa- tional Linguistics.
Jason M. Fisner. 1996. Three new probabilistic models for dependency parsing: An exploration. In COLING 1996 Volume 1: The 16th International Conference on Computational Linguistics.
Joakim Nivre. 2003. An efficient algorithm for pro- jective dependency parsing. In Proceedings of the Eighth International Conference on Parsing Tech- nologies, pages 149-160, Nancy, France.
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation, 1755:173
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz

---

Kaiser, and Illia Polosukhin. 2017. Attention is all you need.
and the 11th International Joint Conference on Natu- ral Language Processing (Volume 1: Long Papers). pages 3946-3957, Online. Association for Computa- tional Linguistics.
Petar Veličković. Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph attention networks. In International Conference Learning Representations.
Longyin Zhang, Yuqing Xing, Fang Kong, Peifeng Li, and Guodong Zhou. 2020. A top-down neural archi- tecture towards text-level parsing of discourse thetor- ical structure. Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guisties, pages 6386-6395, Online. Association for Computational Linguistics.
Yizhong Wang, Sujian Li, and Houfeng Wang. 2017. two-stage parsing method for text-level discourse analysis. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Vol- ume 2: Short Papers), pages 184-188, Vancouver, Canada, Association for Computational Linguistics.
Yifei Zhou and Yansong Feng. 2022. Improve discourse dependency parsing with contextualived representa- tions. In Findings of the Association for Computa- rional Linguisties: NAACL 2022, pages 2250-2261, Seattle, United States. Association for Computational Linguistics.
Jiacheng Xu, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020. Discourse-aware neural extractive text sum- marization. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 5021-5031, Online, Association for Computa- tional Linguistics.
An Yang and Sujian Li. 2018. SciDTB:
Discourse de- pendeney TreeBank for scientific abstracts. In Pro- ceedings of the 56th Annual Meeting of the Associa- tion for Computational Linguistics (Volume 2: Short Papers), pages 444-449, Melhourne, Australia. As- sociation for Computational Linguistics.
Cheng Yi, Li Sujian, and Li Yueyuan. 2021. Unify- ing discourse resources with dependency framework. Proceedings of the 20th Chinese National Con- ference on Computational Linguistics, pages 1058- 1065, Huhhot, China. Chinese Information Process- ing Society of China.
Nan Yu, Meishan Zhang, and Guohong Fu. 2018. Transition-biis based neural RST parsing with implicit syntax features. Proceedings of the 27th Inter- national Conference on Computational Linguistics, pages 559-570, Santa Fe, New Mexico, USA. Asso- ciation for Computational Linguistics.
Nan Yu, Meishan Zhang, Guohong Fu, and Min Zhang 2022. RST discourse parsing with second-stage EDU-level pre-training. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), pages 4269- 4280, Dublin, Ireland. Association for Computational Linguistics.
Liwen Zhang, Ge Wang, Wenjuan Han, and Kewei Tu. 2021a. Adapting unsupervised syntactic pars- ing methodology for discourse dependency parsing. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lan- guage Processing (Volume 1: Long Papers), pages 5782-5794, Online. Association for Computational Linguistics.
Longyin Zhang, Fang Kong, and Guodong Zhou. 202 Adversarial learning for discourse rhetorical structure parsing. Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics


Reviews:
[{"review_id": "MzTMYNgbTQ", "reviewer": "Reviewer_Hmaa", "summary": "", "questions": "A: The authors acknowledge that discourse dependency parsing \"receives relatively less attention\" than discourse constituency parsing. What is the reason for that? Is there a reason why discourse dependency parsing is still attractive?\n\nB: As the authors acknowledge, the results in Table 8 show that taking graph structure into account gives increasing improvements for long-distance connections between EDUs up until the final category of EDUs with distance >18 between them. Is there a hypothesis for what is happening in this category? Is the data noisy or does the model hit some fundamental scaling issues?\n\nC: Are relationships between specific sentences in different paragraphs modelled in this approach?", "limitations": "", "rating": 3, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your efforts and time in our paper, which we believe is crucial to improve our work. \n\nFor hyperparameters, we will give a detailed description in the future version. Here we present several important ones:\n\n(1)\tThe dimension of token, EDU, and sub-tree embeddings is set to 768. In Equation (4), we set d_in to 768 and d_out to 768 / H, where H represents the number of attention heads.\n\n(2)\tFor GATs, we explore different settings for the number of layers (L) chosen from {1, 2, 3, 5}, the number of attention heads (H) selected from {1, 3, 6, 12}, batch size (bs) from {8, 16, 32}, as well as the learning rate and the number of training epochs.\n\na)\tIn the E2S structure stage: L=2, H=3, bs=16, lr = 2e-5, epochs=5\n\nb)\tIn the S2P and P2D structure stage: L=3, H=6, bs=8, lr=2e-5, epochs=5\n\nc)\tIn the relation prediction stage (EGATs): L=3, H=3, bs=32, lr=1e-6 for BERT parameters and 1e-4 for the others, epochs=10. \n\nFor Question (a), section 3.1.1 and 3.1.2 are closely related and can be merged into one section. We put \"Transition State Representations\" and \"EDU Representations\" in section 3.1.2 because we consider them as part of the sub-tree encoder. Sorry for the confusion. We will clarify in the revised version. \n\nFor question (b), our approach involves the following steps:\n\n(1)\tselect the top two EDUs from the stack and the first two EDUs from the queue. \n\n(2)\tconstruct a sub-tree for each source EDU with the arcs obtained in previous steps\n\n(3)\tderive representations for source EDUs based on their respective sub-trees using GATs.\n\n(4)\tconcatenate the representations and employ a fully-connected layer for predicting the subsequent action. \n\nWe will add a detailed algorithm and release our code for further study. \n\nFor Question A, discourse constituency parsing is more popular because it can capture the structure of a discourse more accurately, primarily attributed to the presence of interior nodes representing text spans. On the other hand, discourse dependency parsing can be helpful for downstream tasks, especially information extraction tasks. For instance, it becomes particularly valuable when handling implicit temporal relation recognition tasks, as the length of the path between EDUs may play a crucial role in such scenarios.\n\nFor Question B, the star-graph model demonstrates effective performance on extremely long texts. This success can be attributed to the direct connection of each EDU with the source EDU, enabling the extraction of comprehensive global information. However, the full-graph model performs even worse than no-graph, possibly due to \n\n(1) Limited Training Data: The long texts in the dataset are insufficient to train the model.\n\n(2) Increased Complexity: As the distance between EDUs increases, the model must make a greater number of accurate parsing decisions.\n\n(3) Information Transfer Challenges: The longest discourse contains 303 EDUs, 162 sentences, 45 paragraphs. Simple GATs may face difficulties in effectively transferring information within a complex tree with too many nodes. \n\nFor Question C, we do not consider the relationships between specific sentences in different paragraphs except for the root sentences. We follow the hierarchical structure in which a sub-tree is constructed for each paragraph and they are subsequently merged to form a complete tree. In this arrangement, connections between paragraphs only exist between root EDUs of each paragraph. We adopt this approach for several reasons: \n\n(1)\tThe hierarchical structure reduces the average length of the sequence.\n\n(2)\tOur approach aligns with the target of forming a tree structure.\n\n(3)\tObservations: We define an EDU as an out-EDU if its head is in another paragraph. 89.5% of paragraphs have only one out-EDU (root), and 86.1% of out-EDUs have another out-EDU as its head. These results are acceptable due to the relatively loose connections between paragraphs. These observations encourage us to employ this simplified approach.\n\nIn our future work, we will consider employing more complex structure. For example, we may allow data transferring between all pairs of sentences using a complex graph structure, as opposed to the current tree-based approach.\n\nThanks for pointing out those typos, which we will fix in the later version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The technical idea makes sense and the experimental results give strong evidence that it works well. Detailed analysis of the results corroborate the hypothesis that the hierarchical prediction method primarily gives benefits over discourse structures that are larger and more complex. These results will contribute to our understanding of discourse dependency parsing and give ideas to other researchers and practitioners.", "reasons_to_reject": "While the technical content of the work appears solid, there are clarity issues which makes it hard for the reader to have confidence they fully understand the contribution, or would be able to replicate the work if they wish to:\n\n- There is no discussion of hyperparameters, which will make it hard to replicate the work. E.g. the dimensions of parameter matrices are not given; authors state that they apply \"multiple EGAT layers to update the representations of nodes and edges\" but don't say how many is meant by \"multiple\".\n\n- The algorithm used is complex and has many steps. This means that it has to be explained very clearly. I felt the explanation was not as clear as it could be, even after reading through Section 3 multiple times - I'm sure that if I tried to replicate I would get some details wrong. An appendix containing a fully-worked step-by-step example would be a helpful addition. Some other suggestions:\n\n(a) Make it clear whether \"Transition State Representations\" and \"EDU Representations\" as described in Section 3.1.2 are relevant only to Section 3.1.2 (as the paper structure suggests) or also Section 3.1.1 (which I am guessing is the case)\n\n(b) State explicitly the decision function that is used to classify the next parser action: what are its parameters and arguments?\n\n- A commitment to release code would also help with reproducibility\n", "questions_to_authors": "A: The authors acknowledge that discourse dependency parsing \"receives relatively less attention\" than discourse constituency parsing. What is the reason for that? Is there a reason why discourse dependency parsing is still attractive?\n\nB: As the authors acknowledge, the results in Table 8 show that taking graph structure into account gives increasing improvements for long-distance connections between EDUs up until the final category of EDUs with distance >18 between them. Is there a hypothesis for what is happening in this category? Is the data noisy or does the model hit some fundamental scaling issues?\n\nC: Are relationships between specific sentences in different paragraphs modelled in this approach?", "paper_topic_and_main_contributions": "This paper presents a hierarchical prediction method for discourse dependency parsing of documents. Intuitively, a sentence contains a set of discourse units that have discourse relations between them; a paragraph contains a set of sentences that have discourse relations between them; a text contains a set of paragraphs that have discourse relations between them. The key method proposed here is to use a rich learned representation of substructures when predicting relations at higher levels, rather than simple heuristic relations used in prior work. Experiments on multiple standard discourse parsing datasets show that this method outperforms previously proposed methods, and deeper analysis illustrates how specific design decisions contribute to this superiority.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "p.1, line 44: space in \"distance .\"\n\np.1, line 45: typo \"levles\"\n\np.3, line 158: space in \"2022) ,\"\n\np. 4 \"LeakyReLU\", \"ELU\", v_i' are not defined; please give definitions or citations\n\np.8, line 571: I think \"D2P\" should be \"P2D\" as used elsewhere in the paper"}}, {"review_id": "dl1d5gnusL", "reviewer": "Reviewer_m6QC", "summary": "", "questions": "Nothing particular, but comments on the weakness above could be helpful.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the useful suggestions which will definitely improve our work. \n\nRegarding the lack of statistical evidence, most of the results reported in our paper are the average of six separate runs. We provide both the mean values and their corresponding standard deviations for the experiments shown in Table 2 and Table 4 here: \n\nTable 2 :  Discourse dependency parsing results on SciDTB (with 26 fine-grained relations) and CDTB (with 18 relations) : \n\n|  model   | SciDTB_UAS |   SciDTB_LAS |  CDTB_UAS |  CDTB_LAS |\n\n| Zhou22\t| 78.08 ± 1.06 | 62.42 ± 0.72 | 81.76 ± 1.18 | 62.63 ± 0.87 |\n\n| Ours       | 81.28 ± 0.34 | 67.31 ± 0.44 | 84.36 ± 0.05 | 67.61 ± 0.36 |\n\nTable 4 : Structure prediction results on SciDTB : \n\n| model          | Intra sentence   |   inter_p  |     inter_g |\n\n| No-Graph | 87.45 ± 0.86 | 63.21 ± 1.49 | 66.54 ± 1.30 |\n\n| Star-Graph | 88.85 ± 0.70 | 65.36 ± 1.28 | 68.73 ± 1.13 |\n\n| Full-Graph | 89.09 ± 0.35 | 67.81 ± 0.88 | 71.16 ± 0.66 |\n\nIn Table 2, it becomes evident that our model exhibits a significant performance advantage over Zhou22. Furthermore, Table 4 reveals remarkable distinctions between No-Graph and Star-Graph, as well as between Star-Graph and Full-Graph, particularly at the inter-sentence level.\n\nWe will conduct a significance test for the remaining experiments in a forthcoming version of this study.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The model proposed in the paper is elaborately described. Empirical results and the corresponding analysis are sufficient to illustrate the superiority of the proposed model. ", "reasons_to_reject": "The authors claimed the proposed model achieved a significant improvement without providing any statistical evidence. The authors should provide several proper statistics to quantify the variability in the performance of their proposed model.", "questions_to_authors": "Nothing particular, but comments on the weakness above could be helpful.", "paper_topic_and_main_contributions": "The paper used a GAT-based encoder to dynamically represent sub-trees constructed in transition history. Further, the paper proposed a novel parsing model based on the representations produced by the GAT-based encoder. The experimental results showed the superiority of the proposed model on multiple text corpora. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "None", "Typos_Grammar_Style_and_Presentation_Improvements": "line 023: resarch -> research\nline 046: levles -> levels\nline 161: make full of -> make full use of"}, "extra_text": {}}, {"review_id": "aeMk6pww2R", "reviewer": "Reviewer_pLZG", "summary": "", "questions": "- A: It seems that at each transition-step, the EDU representations are re-encoded with the updated structures? Is this necessary, and are there any incremental solutions the authors may have considered?\n- B: Instead of the pipelined approach that separately predicts the structures and the labels, would it be beneficial to predict them jointly?\n- C: Are there any specific reasons to adopt the arc-eager system? How about other transition systems, such as arc-standard?\n", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your time and helpful suggestions in our work. \n\nFor the question of baselines, there are few works on the discourse dependency parsing task, and most early works do not use pre-trained language models. Therefore we only select two baselines, which are the SOTA transition-based and graph-based models. Our model and the two baselines employ the same PLM (BERT-base) for fair comparison.\n\nFor the question of ablation study, it's important to note that 'No-Graph' in Table 4 maintains an identical structure to Zhou22, with only minor modifications in the encoding component.  \nThe results presented in Table 4 clearly demonstrate the advantages of incorporating EDUs within the sub-tree structure. Specifically, 'Star-Graph' outperforms 'No-Graph,' highlighting the benefits of including EDUs. Moreover, the introduction of a sub-tree structure further enhances performance, as evidenced by 'Full-Graph' surpassing 'Star-Graph’. Besides, Table 4 and 5 illustrate the performance of our model at different parsing levels, and Table 6, 7 and 8 show the performance of our model on long texts.\n\nFor the question of efficiency, we admit that our model incurs significant computational costs. This becomes particularly noticeable when dealing with longer texts, especially during the P2D stage in the RST-DT dataset. However, it manages to achieve a much better performance in SciDTB and CDTB with an acceptable increase in processing time.\n\nWhen comparing our full-graph model to the no-graph version (similar to Zhou22, which uses the representation of a single root EDU) at both E2S and S2P levels on SciDTB and CDTB, the time requirements for each transition step during training are as follows: \n\nE2S Level:\n\nSciDTB: 0.063s (full-graph) vs. 0.051s (no-graph)\n\nCDTB: 0.067s (full-graph) vs. 0.049s (no-graph)\n\nS2P Level:\n\nSciDTB: 0.206s (full-graph) vs.0.063s (no-graph)\n\nCDTB: 0.195s (full-graph) vs. 0.056s (no-graph)\n\nNotably, at the E2S level, both versions demand a similar amount of time, making both of them efficient. Even at the S2P level, where the processing time increases, it remains within tolerable limits. Consequently, our method is well-suited for a wide range of applications, except for cases involving exceptionally long texts where efficiency concerns may arise.\n\nFor Question A, the EDU representations are re-encoded after each step. We believe this is beneficial to update the EDU representation with different sub-tree structures although it is time costly. In our future research, we plan to explore the possibility of employing an approximate method for updating EDU representations within a sub-tree. This method would involve merging the representations of two connected sub-trees rather than completely re-encoding them from scratch.\n\nFor Question B, the datasets for the discourse dependency parsing task are relatively small. The available data within each class may be too sparse if we attempt to predict both the structure and relations jointly.\n\nRegarding Question C, we think the arc-standard system may be inferior to the arc-eager system, because the latter permits an EDU to establish a connection with its head earlier in the parsing process, bypassing the need to wait for all of its children to be parsed, which may be helpful for long texts. We will perform experiments on this argument in our future work. Thank you for your advice.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Modeling partial structures during the parsing procedure is an intuitive idea and it is nice to see that it works for the discourse-level parsing task.\n- The proposed model is reported to obtain better results than previous methods.\n", "reasons_to_reject": "- There are few baselines that are compared and ablation studies are lacking. Though there may be few works on this task, it would be nice to provide more analysis to illustrate which model designs bring the most improvements. Especially, “Zhou22” is also a hierarchical transition-based model and would be a potentially good start point to perform the ablations.\n- Although mentioned in the Limitation section, there is no analysis on the efficiency of the proposed model. Encoding the sub-tree structures may bring large extra costs, which should be carefully considered to better apply the proposed model.\n", "questions_to_authors": "- A: It seems that at each transition-step, the EDU representations are re-encoded with the updated structures? Is this necessary, and are there any incremental solutions the authors may have considered?\n- B: Instead of the pipelined approach that separately predicts the structures and the labels, would it be beneficial to predict them jointly?\n- C: Are there any specific reasons to adopt the arc-eager system? How about other transition systems, such as arc-standard?\n", "paper_topic_and_main_contributions": "This work proposes a hierarchical transition-based model for the task of discourse dependency parsing. The parsing procedure is performed with multiple levels, and the partial structures obtained from previous transitions are models with a GAT-based encoder to leverage parsing history. A similar GAT-based model is utilized to predict discourse relations. With evaluations on three datasets, the proposed models are shown to perform better than previous methods and other baselines.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "L046: levles -> levels"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a method relying on EGAT (edge graph attention networks) for hierarchical discourse dependency parsing. Several experiments and comparisons are reported showing that the method outperforms comparable methods on several standard corpora. Further analysis shows that the method is good for complex longer discourse dependencies/structures. All the reviewers agree on these. 

The paper contributes to the field because there's relatively few works on discourse dependency parsing in general, or models that . Besides, the experimentation is good, sound and broad. However, there's an incremental feeling to the main ideas in that they basically adapt some existing methodology to an existing problem.
Decision: Accept-Findings