Context-Aware Word Segmentation for C hinese Real-World Discourse (2020)

Abstract: Previous neural approaches achieve significant progress for Chinese word segmentation (CWS) as a sentence-level task, but it suffers from limitations on real-world scenario. In this paper, we address this issue with a context-aware method and optimize the solution at document-level. This paper proposes a three-step strategy to improve the performance for discourse CWS. First, the method utilizes an auxiliary segmenter to remedy the limitation on pre-segmenter. Then the context-aware algorithm computes the confidence of each split. The maximum probability path is reconstructed via this algorithm. Besides, in order to evaluate the performance in discourse, we build a new benchmark consisting of the latest news and Chinese medical articles. Extensive experiments on this benchmark show that our proposed method achieves a competitive performance on a document-level real-world scenario for CWS.

## introduction downstream tasks in chinese natural language processing (nlp) leverage word-level information to construct architectures. in recent years, some technologies gradually replace the word-level information in order to alleviate segmentation errors and word sparse problems @xcite . however, it may lose all of the other word-level information (e.g., part-of-speech, and dependency parsing). chinese word segmentation (cws) is still essential for downstream chinese nlp tasks. @xcite formalized cws task as a sequence labeling problem. the performance for cws has achieved significant progress via statistical machine learning @xcite and neural network methods @xcite @xcite . in particular, recent years have also seen * corresponding author a new supervised learning paradigm in applying bert or other pre-training models for sequence labeling problems @xcite @xcite . various fine-tuning methods can improve the performance for in-domain cws significantly and easily. previous researches almost perform like humans with a nearly 2% error rate via pre-training methods. and neural methods do not rely on the hand-craft feature engineering, compared with statistical machine learning methods. however, recent state-of-the-art neural methods always suffer from two limitations as follow: both two limitations are reflected under real-world document-level texts apparently. many chinese tasks are discourse processing researches in most cases, e.g., text classification and machine translation. so the effectiveness for real-world documentlevel cws affects the performance on these tasks. real-world texts are always time-efficient. not only cross-domain but also time-validity leads to the issue of out-of-vocabulary (oov) words. even though previous researches alleviate the issues for cross-domain cws, the issue for time-efficient cws is hard to solve with similar inspiration by these effective methods. most methods start with data-driven to improve the performance for crossdomain cws. for instance, previous works incorporate the domain dictionary and pre-training embedding into neural network methods @xcite @xcite . annotating a small number of training corpora is the most effective and simple method. however, large annotated corpus cannot be updated immediately to deal with the latest news and dialogues, due to the high cost of the hand-craft annotated work. similarly, training an effective pre-training model is time-consuming. it is impractical to update the model and annotated corpus with the latest data. furthermore, previous effective methods always suffer from the weakness of robustness for discourse cws, as shown in figure 1 . the word "gao shan" occurs twice with the context of similar semantics in the discourse. however, it is segmented into different splits. because of the limitation on the maximum length of input sequences, segmentation consistency is not guaranteed. previous works for in-domain and cross-domain cws are always character-level and sentence-level. @xcite proposes an effective method for patent domain cws via integrating the documentlevel features, but it is still a sentence-level optimization . @xcite utilizes multiple constraint rules to alleviate the issues on specific domains. this paper proposes a context-aware unsupervised method to alleviate the above issues for chinese word segmentation, instead of adopting multiple constraint rules. and it is not limited to a specific domain. the method is aware of a global receptive field in the entire discourse. it utilizes the document-level information directly to improve the performance for cws on the real-world discourse scenario. in particular, the words that recur in discourse are rejudged by our proposed method. the uncertain words are also reconsidered. the method consists of three steps: a word-lattice based pre-segmenter, a rejudged module, and a context-aware algorithm. first, the pre-segmenter achieves high performance on in-vocabulary words. then the rejudged module chooses the uncertain splits as potential out-of-vocabulary words. finally, the core context-aware algorithm utilizes the document-level information to screen the uncertain splits. the sequence labeling task is to find the maximum probability path. a new path is reconstructed through the three steps. to evaluate our method, we build a new benchmark with document-level texts for cws. it contains the latest news and chinese medical articles. extensive experiments show that our proposed method is effective in discourse area and achieves a competitive performance for real-world cws. to sum up, our main contributions are threefold: 1) to the best of our knowledge, our proposed method is the first work to adopt a document-level unsupervised learning algorithm for cws in a realworld scenario. it only takes the information about the current discourse itself. the benefit is that there is no need to do maintenance work for external resources constantly. 2) the method acts on the global field of discourse. it can alleviate the issue of segmentation inconsistency effectively. 3) we propose a new benchmark to evaluate the performance for cws on real-world discourse scenario. foot_0 ## methodology figure 2 shows the entire process of our proposed method. it consists of three steps. in the first two steps, we utilize a pre-segmenter and an auxiliary segmenter to segment the sentences. the two segmenters generate two segmentation results r and r respectively. it can distribute segmentation splits with different reliability. the pre-segmenter and auxiliary segmenter are all sentence-level cws methods. in the third step, the context-aware algorithm leverages the document-level information to determine the final boundary of these splits. it is global optimization to revise the results in the prior steps. the final segmentation result r o is obtained by this optimization. ## conclusion in this paper, we intuitively propose a contextaware method to boost the segmentation inconsistency in discourse. the time-efficient and domain knowledge are considered via the document-level information. the method is explainable and unsupervised. in summary, our proposed method is empirical but do not used stiff constraint rules. besides, a new benchmark in discourse is built for evaluation of the document-level chinese word segmentation. the distribution of words is natural on benchmark. however, the scale of the benchmark is still limited. we will expand them and make it more reliable. and we will try to integrate the knowledge into popular neural models in the future.