A Joint Model for Graph-based C hinese Dependency Parsing (2020)

Abstract: In Chinese dependency parsing, the joint model of word segmentation, POS tagging and dependency parsing has become the mainstream framework because it can eliminate error propagation and share knowledge, where the transition-based model with feature templates maintains the best performance. Recently, the graph-based joint model (Yan et al., 2019) on word segmentation and dependency parsing has achieved better performance, demonstrating the advantages of the graph-based models. However, this work can not provide POS information for downstream tasks, and the POS tagging task was proved to be helpful to the dependency parsing according to the research of the transition-based model. Therefore, we propose a graph-based joint model for Chinese word segmentation, POS tagging and dependency parsing. We designed a charater-level POS tagging task, and then train it jointly with the model of Yan et al. (2019). We adopt two methods of joint POS tagging task, one is by sharing parameters, the other is by using tag attention mechanism, which enables the three tasks to better share intermediate information and improve each other’s performance. The experimental results on the Penn Chinese treebank (CTB5) show that our proposed joint model improved by 0.38% on dependency parsing than the model of Yan et al. (2019). Compared with the best transition-based joint model, our model improved by 0.18%, 0.35% and 5.99% respectively in terms of word segmentation, POS tagging and dependency parsing.

## introduction chinese word segmentation, part-of-speech (pos) tagging and dependency parsing are three fundamental tasks for chinese natural language processing, whose accuracy obviously affects downstream tasks such as semantic comprehension, machine translation and question-answering. the traditional method is usually following pipline way: word segmentation, pos tagging and dependency parsing. however, there are two problems of the pipline way, one is error propagation:incorrect word segmentation directly affects pos tagging and dependency parsing, another is information sharing: the tree tasks are strongly related, the label information of one task can help others, but the pipline way cannot exploit the correlations among the three tasks. using joint model for chinese word segmentation, pos tagging and dependency parsing is a solution to these two problems. the previous joint models @xcite @xcite ) mainly adopted a transition-based framework to integrate the three tasks. based on the standard sequential shift-reduce transitions, they design some extra actions for word segmentation and pos tagging. although these transition-based models maintained the best performance of word segmentation, pos tagging and dependency parsing, its local decision problem led to the low precision of long-distance dependency parsing, which limited the precision of dependency parsing. different from the transition-based framework, the graph-based framework has the ability to make global decisions. before the advent of neural network, the graph-based framework was rarely applied to the joint model due to its large decoding space to calculate.with the development of neural network technology, the graph-based method for dependency parsing improves rapidly and comes back into researchers' vision. yan et al. (2019) firstly proposed a graph-based unified model for joint chinese word segmentation and dependency parsing with neural network and attention mechanism, which is superior to the best transition-based joint model in terms of word segmentation and dependency parsing. this work without pos tagging task shows that dependency parsing task is beneficial to chinese word segmentation. chinese word segmentation, pos tagging and dependency parsing are three highly correlated tasks and can improve each other's performance. dependency parsing is beneficial to word segmentation and pos tagging, while word segmentation and pos tagging are also helpful to dependency parsing, which has been demonstrated by considerable work on the existing transition-based joint model of three tasks. we consider that joint pos tagging task can further improve the performance of dependency parsing. in addition, it makes sense of the model to provide pos information for downstream tasks. for these reasons, this paper proposes a graph-based joint model for word segmentation, pos tagging and dependency parsing. first, we design a character-level pos tagging task, and then combine it with a graph-based joint model for word segmentation and dependency parsing @xcite . as for the joint approach, this paper proposes two ways, one is to combine the two tasks by hard sharing parameters @xcite and the other is combine the two tasks by introducing tag attention mechanism in the shared parameter layer. finally, we analyze our proposed models on the chinese treebank (ctb5) dataset. ## the proposed model in this section, we introduce our proposed graph-based joint model for chinese word segmentation, pos tagging and dependency parsing. through the joint pos tagging task, we explore the joint learning method among multiple tasks and seek for a better joint model to improve the performance of chinese dependency parsing further.