C hinese and E nglish Elementary Discourse Units Segmentation based on B i- LSTM - CRF Model (2020)

Abstract: Elementary Discourse Unit (EDU) recognition is the basic task of discourse analysis, and the Chinese and English discourse alignment corpus is helpful to the studies of EDU recognition. This paper first builds Chinese-English parallel discourse corpus, in which EDUs are annotated and aligned. Then, we present the framework of Bi-LSTM-CRF EDUs recognition model using word embedding, POS and syntactic features, which can combine the advantage of CRF and Bi-LSTM. The results show that F1 is about 2% higher than the traditional method. Compared with CRF and Bi-LSTM, the Bi-LSTM-CRF model can combine the advantages of them and obtains satisfactory results for Chinese and English EDUs recognition. The experiment of feature contribution shows that using all features together can get best result, the syntactic feature outperforms than other features.

## introduction discourse analysis is helpful for the performance of machine translation, question answering, summarization and other application. edu recognition is a basic work in discourse analysis task. only by recognition edu, can we make further discourse analysis or other works. at present, the existing chinese-english parallel corpus only align paragraphs, sentences and other linguistic units, but do not annotate bilingual edus alignment, which due to edu recognition is mainly carried out on monolingual. however, edus recognition on chinese and english is vital to bilingual analysis, machine translation et al. for example1 is a bilingual sentence of chinese and english, chinese edus are numbered sequentially by e1, e2 and e3, and english edus are marked by e1 , e2 and e3 . obviously, e1 and e1 , e2 and e2 , e3 and e3 are alignment pair. example 1 a) [®é$à 5"u §] e1 [üwû_ 8 §] e2 [n´uˆ"] e3 b) [the beijing -hangzhou grand canal has been prosperous since ancient times,] e1 [with both sides of the bank swarming with merchants] e2 [and well -developed trade.] e3 the main work of this paper is recognition the edus of chinese and english as much as possible. the following is the contribution of this paper: we annotate chinese and english discourse alignment corpus, which is first corpus contain edus alignment information as far as we know; we conduct to find out the contribution of various model and features. this paper combines existing research and chinese-english discourse alignment corpus to identify and analyze chinese-english edus. section 2 builds chinese-english edus alignment corpus; section 3 describes the bi-lstm-crf model and the framework this paper used; section 4 reports and analyzes the experimental results; section 5 overviews the related work; finally, section 6 summarizes this paper and points out the future research direction. proceedings of the 19th china national conference on computational linguistics, pages 1068-1078, hainan, china, october 30 -novermber 1, 2020. (c) technical committee on computational linguistics, chinese information processing society of china c c l 2 0 2 0 ## the model of edu recognition based on bi-lstm-crf in this section, we introduced the bi-lstm-crf model we used, which is the combination of crf and bi-lstm and have been used in several nlp task. ## conclusion the discourse alignment corpus of chinese-english is annotated in this paper. the corpus has a complete edu definition, annotation method, quality assurance and available scale. the corpus we annotated in this paper is the basic task of edus recognition. then we developed an edus recognition system using bi-lstm-crf model. our neural model achieved satisfactory results for chinese and english edu recognition. to our knowledge, we are among the first to develop an effective neural network-proceedings of the 19th china national conference on computational linguistics, pages 1068-1078, hainan, china, october 30 -novermber 1, 2020. (c) technical committee on computational linguistics, chinese information processing society of china c c l 2 0 2 0 based approach to recognize edus for both chinese and english. we input word embedding, pos and syntactic feature to our model in order to improve the result. by incorporating these features, our model can extract edus automatically and high quality. the f1 can reach 93.4% and 94.4% for chinese and english separately, which is reaching the practical using. this model can also be generalized to solve other problems. in the future, we will improve the effect of recognition chinese and english edus, then try to automatic align them.