SlowGuess/ACL_2020_OCR huggingface dataset:
We use the word embedding representation based on the 300D vectors provided by Glove (2014)<sup>1</sup> and Qiu(2018) for English and Chinese respectively, and do not update the weights of these vectors during training, while the POS embedding uses the random initialization method and is optimized with our model. We fine-tune the hyper-parameters on the development set as shown in Table 1.
4.2.1 Overall Performance
First, Table 2 compares the detailed performance of our top-down discourse parser with the state-of-the-art on gold standard EDUs.
For English RST-style text-level discourse parsing, we evaluate our top-down discourse parser on the RST-DT corpus and compare our model with five state-of-the-art systems as mentioned in Morey (2018) using the same evaluation metrics.
- Ji and Eisenstein (2014), a shift-reduce parser that learns the representation of discourse units
1Impact of other pre-trained word embedding is limited. For example, ELMo can improve the full-score about  $0.6\%$ .
2We evaluate the discourse parsers proposed by Lin et al. (2019) and Liu et al. (2019) in text-level discourse parsing. However, their achieved performances are much lower than the state-of-the-art systems. The main reason is that their proposed encoders are tailored to small text spans in sentence-level discourse parsing and are not suitable for large text spans in text-level discourse parsing. In following experiments, we no longer compare our system with them.
6391
and trains an SVM classifier jointly with a lot of hand-crafted features.
- Feng and Hirst (2014), a two stage greedy parser with linear-chain CRF models.
- Li et al. (2016), an attention-based hierarchical model along with hand-crafted features.
- Braud et al. (2016), a sequence-to-sequence parser that is heuristically constrained to build trees with a hierarchical neural model.
- Braud et al. (2017), a transition-based neural model with a lot of cross-lingual features.
For Chinese CDT-style text-level discourse parsing, there are much fewer studies. Sun and Kong (2018) propose a complete transition-based Chinese discourse structure generation framework. However, they only concerned tree structure generation and did not consider discourse relation classification. In fact, just as noted in Wang et al. (2017), a transition-based model is more appropriate for parsing the bare discourse tree structure due to the data sparsity problem. In addition, since relation classification can benefit from the bare tree structure, a two stage parsing strategy can normally achieve better performance. In comparison, with the support of local contextual information of split points and global high-level discourse structure information, our top-down architecture is able to identify the discourse structure and discourse relations jointly. For fair comparison, we duplicate the approach proposed by Sun and Kong (2018), and evaluate it under the same experimental settings<sup>3</sup>. We call this system as the duplicated system (denoted as "Dup"). Table 2 shows that,
- For English, our top-down system achieves comparable performance with the state-of-the-art systems. It is worthwhile to note that, we focus on the effectiveness of our proposed top-down architecture in this paper. The performance of our top-down system is achieved without any other additional features, while other systems employ
${}^{3}$  Sun and Kong (2018) reported their performance using macro-averaged  ${F}_{1}$  scores. In fact, it increases the weight of shorter documents. For Chinese CDTB, each paragraph is represented as a CDT. Statistics on the distribution of CDT heights shows that, one CDT contains about 4.5 EDUs on average, with the average height about 3.42. In this paper, we report the performance using micro-averaged  ${F}_{1}$  scores. Furthermore, to gain detailed comparison between the bottom-up and the top-down approaches, we also report the performance of relation classification and full discourse parsing.
language	Bare	Nuclearity	Relation	Full
EN	62.3	50.1	40.7	39.6
CN	80.2	53.2	48.5	41.7
Table 3: Performance under a full automatic setting.
various additional features. For example, both Ji and Eisenstein (2014) and Feng and Hirst (2014) employed many kinds of additional hand-crafted features including syntactic, contextual and so on, while Braud et al. (2017) resort to additional cross-lingual features and achieve the gain of 3.2, 7.3, 10.8 and 10.8 on the four evaluation metrics respectively in comparison with Braud et al. (2016). This indicates the great preference of top-down over bottom-up text-level DRS parsing. This also suggests the great potential of additional carefully designed features, which are worth exploring in the future work.
- For Chinese, our top-down text-level DRS parser significantly outperforms Sun and Kong (2018) on bare DRS generation, nuclearity determination and relation classification with all p-values smaller than 0.01 on signcate testing. However, we find that our top-down approach achieves relatively poor performance on Full discourse parsing. This maybe due to the effectiveness of the joint learning framework as employed in Sun and Kong (2018). Traditional shift-reduce approaches cast the parsing task as a triple (i.e., shift/reduce action, nuclearity and relation type) identification task, and learn/predict the triple simultaneously, while our top-down approach divides the discourse parsing task into three independent sub-tasks, i.e., split point ranking, nuclearity determination and relation classification, and optimize our discourse parsing model only using the Negative Log Likelihood Loss. This also applies to the English discourse parser discussed above.
- Comparing the results for English and Chinese, Chinese text-level discourse parsing looks better on all performance metrics. This maybe due to the difference between annotation strategies. In English RST-DT corpus, each document is represented as one DT, while in Chinese CDTB, each paragraph is represented as a CDT. As a result, the CDTs generally contain fewer EDUs and are relatively short in height.
6392
Height	Std	Bare	Nuc	Rel	Full
↓	↑	↓	↑	↓	↑	↓	↑
1	385	339	321	251	221	233	215	213	200
2	220	183	184	117	115	116	111	94	101
3	139	119	122	71	82	71	73	59	71
4	88	75	78	52	58	44	42	39	40
5	44	34	37	17	21	16	21	10	16
6	26	18	21	13	13	6	9	6	9
7	18	16	18	7	8	6	9	2	5
>= 8	13	11	10	0	0	0	0	0	0
Overall	933	795	791	535	521	497	486	426	445
4.2.2 End-to-end Performance
Next, Table 3 shows the performance of the end-to-end text-level discourse parser under a full automatic setting. Here, we use the two EDU detectors proposed by Li et al. (2018) and Li et al. (2013) to achieve the auto EDUs for English and Chinese respectively, and the berkeley parser<sup>4</sup> to achieve automatic parse trees. From the results shown in Table 3 we can find that, in comparison with the overall performance using gold standard EDUs shown in Table 2, there is a significant performance reduction on all the indicators. This indicates the heavy impact of EDU segmentation.
4.2.3 Detailed Analysis
Finally, we take Chinese as an example for a detailed comparative analysis. We duplicate the approach proposed by Sun and Kong (2018) and take this duplicated system as the representative of the bottom-up approach.
Table 4 first compares the results over different DT levels with the gold standard numbers and the correctly identified numbers. It should be noted that, correctly determined nuclearity means both the bare tree node and its nuclearity are correctly recognized. Correctly determined relation means both the bare node and its relation are correctly recognized, and full means all three aspects are correctly recognized. From the results we can find that, in comparison with the bottom-up approach, the top-down approach can achieve better performance on Bare, Nuc and Rel metrics, while for Full-metric, the performance reduces slightly. Just as noted above, this is due to the difference between the joint learning frameworks behind these two approaches. Among three aspects, the improvement of nuclearity is most, and bare tree structure is weakest. At each level, the performance of these
Table 4: Performance over different DT levels. ("↓" - Top down approach, "↑" - Bottom up approach)
Approach	NN	NS	SN
↓	67.0	42.2	33.7
↑	67.6	35.4	24.5
Table 5: Performance on nuclearity determination.
EDU Num	Bare	Nuc	Rel
↑	1-5	94.8	57.9	52.0
6-10	87.0	60.7	58.6
11-15	78.0	50.1	45.4
16-20	56.2	25.0	25.0
21-25	68.9	47.0	42.4
26-30	65.4	26.9	11.5
↓	1-5	97.0	67.1	56.6
6-10	86.0	57.3	59.9
11-15	75.2	50.3	41.4
16-20	56.2	25.0	25.0
21-25	76.6	57.7	40.8
26-30	69.2	42.3	19.2
Table 6: Performance over different EDU numbers.
two approaches varies. This suggests that the bidirectional architecture may be an important direction in the future work.
Since the improvement of nuclearity is significant, we then list the detailed results of these two approaches over different nuclearity categories. Table 5 shows that our top-down approach can determine the "NS" and "SN" much better than the bottom-up approach. This is consistent with human perception.
We finally divide the DTs into six groups by EDU number and evaluate the two approaches over different groups. Table 6 shows the results. We can find that, our top-down approach achieves better performance on the first, fifth and sixth sets (i.e., the EDU number is 1-5, 21-25 and 26-30 respectively). This suggests that the proposed top-down approach may be more suitable for both end of DTs with others comparable.
4https://github.com/slavpetrov/berkeleyparser
6393
In this paper, we propose a top-down neural architecture to text-level discourse parsing. In particular, we cast the discourse parsing task as a EDU split point ranking task, where a split point is classified to different levels according to its rank, and the EDUs associated with the split point are arranged accordingly. In this way, we can determine the complete discourse rhetorical structure as a hierarchical tree structure. Specifically, after encoding the EDUs and EDU split points, a encoder-decoder with an internal stack is employed to generate discourse tree recursively. Experimentation on the English RST-DT corpus and the Chinese CDTB corpus shows the great effectiveness of our proposed approach. In the future work, we will focus on more effective discourse parsing with additional carefully designed features and joint learning with EDU segmentation.
The authors would like to thank the anonymous reviewers for the helpful comments. We are greatly grateful to Cheng Sun for his inspiring ideas and preliminary work. This work is supported by Artificial Intelligence Emergency Project 61751206 under the National Natural Science Foundation of China, Project 61876118 under the National Natural Science Foundation of China and the Priority Academic Program Development of Jiangsu Higher Education Institutions.
Chloe Braud, Maximin Coavoux, and Anders Søgaard. 2017. Cross-lingual RST discourse parsing. arXiv preprint arXiv:1701.02946.
Chloe Braud, Barbara Plank, and Anders Søgaard. 2016. Multi-view and multi-task training of RST discourse parsers. In Proceedings of COLING 2016, pages 1903-1913.
Lynn Carlson and Daniel Marcu. 2001. Discourse tagging reference manual. ISI Technical Report ISI-TR-545, 54:56.
Kyunghyun Cho, Bart van Merrienboer, Caglar Gul- cehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of EMNLP 2014, pages 1724-1734.
Eunsol Choi, Hannah Rashkin, Luke Zettlemoyer, and Yejin Choi. 2016. Document-level sentiment inference with social, faction, and discourse context. In Proceedings of ACL 2016, pages 333-343.
Qianyin Dai, Longyin Zhang, and Fang Kong. 2019. Event temporal relation identification based on dependency and discourse relation.
Timothy Dozat and Christopher D. Manning. 2017. Deep bioaffine attention for neural dependency parsing. In Proceedings of ICLR 2017.
Vanessa Wei Feng and Graeme Hirst. 2014. A linear-time bottom-up discourse parser with constraints and post-editing. In Proceedings of ACL 2014, pages 511-521.
Naman Goyal and Jacob Eisenstein. 2016. A joint model of rhetorical discourse structure and summarization. In Proceedings of the Workshop on Structured Prediction for NLP, pages 25-34.
Michael Heilman and Kenji Sage. 2015. Fast rhetorical structure theory discourse parsing. arXiv preprint arXiv:1505.02425.
Hugo Hernault, Helmut Prendinger, Mitsuru Ishizuka, et al. 2010. Hilda: A discourse parser using support vector machine classification. Dialogue & Discourse, 1(3).
Yangfeng Ji and Jacob Eisenstein. 2014. Representation learning for text-level discourse parsing. In Proceedings of ACL 2014, pages 13-24.
Yangfeng Ji and Noah A. Smith. 2017. Neural discourse structure for text categorization. In Proceedings of ACL 2017, pages 996-1005.
Shafiq Joty, Giuseppe Carenini, Raymond Ng, and Yashar Mehdad. 2013. Combining intra-and multisentential rhetorical parsing for document-level discourse analysis. In Proceedings of ACL 2013, pages 486-496.
Jing Li, Aixin Sun, and Shafiq Joty. 2018. Segbot: A generic neural text segmentation model with pointer network. In IJCAI, pages 4166-4172.
Jiwei Li, Rumeng Li, and Eduard Hovy. 2014a. Recursive deep models for discourse parsing. In Proceedings of EMNLP 2014, pages 2061-2069.
Qi Li, Tianshi Li, and Baobao Chang. 2016. Discourse parsing with attention-based hierarchical neural networks. In Proceedings of EMNLP 2016, pages 362-371.
Sujian Li, Liang Wang, Ziqiang Cao, and Wenjie Li. 2014b. Text-level discourse dependency parsing. In Proceedings of ACL 2014, pages 25-35.
Yancui Li, wenhe Feng, jing Sun, Fang Kong, and Guodong Zhou. 2014c. Building Chinese discourse corpus with connective-driven dependency tree structure. In Proceedings of EMNLP 2014, pages 2105-2114.
6394
Yancui Li, Wenhe Feng, Guodong Zhou, and Kunhua Zhu. 2013. Research of Chinese clause identificaton based on comma. Acta Scientiarum Naturalium Universitatis Pekinensis, 49(1):7-14.
Xiang Lin, Shafiq Joty, Prathyusha Jwalapuram, and M Saiful Bari. 2019. A unified linear-time framework for sentence-level discourse parsing. In Proceedings of ACL 2019, pages 4190-4200.
Linlin Liu, Xiang Lin, Shafiq Joty, Simeng Han, and Lidong Bing. 2019. Hierarchical pointer net parsing. In Proceedings of EMNLP 2019, pages 1006-1016.
William Mann and Sandra Thompson. 1988. Rhetorical structure theory: Toward a functional theory of text organization. Text, 8(3):243-281.
Daniel Marcu. 2000. The Theory and Practice of Discourse Parsing and Summarization. MIT Press.
Mathieu Morey, Philippe Muller, and Nicholas Asher. 2018. A dependency perspective on RST discourse parsing and evaluation. Computational Linguistics, pages 198-235.
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of EMNLP 2014, pages 1532-1543.
Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The penn discourse treebank 2.0. In LREC 2008.
Yuanyuan Qiu, Hongzheng Li, Shen Li, Yingdi Jiang, Renfen Hu, and Lijiao Yang. 2018. Revisiting correlations between intrinsic and extrinsic evaluations of word embeddings. In CCL & NLP-NABD 2017, pages 209-221. Springer.
Cheng Sheng, Fang Kong, and Guodong Zhou. 2017. Towards better Chinese zero pronoun resolution from discourse perspective. In *Processes of NLPCC* 2017, pages 406-418.
Cheng Sun and Fang Kong. 2018. A transition-based framework for Chinese discourse structure parsing. Journal of Chinese Information Processing, 32(12):26-34.
Yizhong Wang, Sujian Li, and Houfeng Wang. 2017. A two-stage parsing method for text-level discourse analysis. In Proceedings of ACL 2017: short paper, pages 184-188.
Nan Yu, Meishan Zhang, and Guohong Fu. 2018. Transition-based neural RST parsing with implicit syntax features. In Proceedings of the 27th International Conference on Computational Linguistics, pages 559-570.
6395