Coupling Distant Annotation and Adversarial Training for Cross-Domain C hinese Word Segmentation (2020)

Abstract: Fully supervised neural approaches have achieved significant progress in the task of Chinese word segmentation (CWS). Nevertheless, the performance of supervised models always drops gravely if the domain shifts due to the distribution gap across domains and the out of vocabulary (OOV) problem. In order to simultaneously alleviate the issues, this paper intuitively couples distant annotation and adversarial training for cross-domain CWS. 1) We rethink the essence of “Chinese words” and design an automatic distant annotation mechanism, which does not need any supervision or pre-defined dictionaries on the target domain. The method could effectively explore domain-specific words and distantly annotate the raw texts for the target domain. 2) We further develop a sentence-level adversarial training procedure to perform noise reduction and maximum utilization of the source domain information. Experiments on multiple real-world datasets across various domains show the superiority and robustness of our model, significantly outperforming previous state-of-the-arts cross-domain CWS methods.

## introduction chinese is an ideographic language and lacks word delimiters between words in written sentences. therefore, chinese word segmentation (cws) is often regarded as a prerequisite to downstream tasks in chinese natural language processing. this task is conventionally formalized as a characterbased sequence tagging problem @xcite , where each character is assigned a specific label to denote the position of the character in a word. with the development of deep learning techniques, recent years have also seen increasing interest in applying neural network models onto cws (cai * corresponding author figure 1 : different word distributions for the newswire domain and the medical domain. @xcite @xcite . these approaches have achieved significant progress on in-domain cws tasks, but they still suffer from the cross-domain issue when they come to processing of out-of-domain data. cross-domain cws is exposed to two major challenges: 1) gap of domain distributions. this is a common issue existing in all domain adaptation tasks. source domain data and target domain data generally have different distributions. as a result, models built on source domain data tend to degrade performance when they are applied to target domain data. generally, we need some labeled target domain data to adapt source domain models, but it is expensive and time consuming to manually craft such data. 2) out of vocabulary (oov) problem, which means there exist some words in the testing data that never occur in the training data. source domain models have difficulties in recognizing oov words since source domain data contains no information on the oovs. figure 1 presents examples to illustrate the difference between the word distributions of the newswire domain and the medical domain. segmenters built on the newswire domain have very limited information to segment domain-specific words like "溶菌酶 (lysozyme)". previous approaches to cross-domain cws mainly fall into two groups. the first group aims to attack the oov issue by utilizing predefined dictionaries from the target domain to facilitate cross-domain cws @xcite @xcite , which are apt to suffer from scalability since not all domains possess predefined dictionaries. in other words, these methods are directly restricted by external resources that are available in a target domain. studies in the second group @xcite attend to learn target domain distributions like word embeddings from unlabeled target domain data. in this approach, source domain data is not fully utilized since the information from source domain data is transferred solely through the segmenter built on the data. in this paper, we propose to attack the aforementioned challenges simultaneously by coupling the techniques of distant annotation and adversarial training. the goal of distant annotation is to automatically construct labeled target domain data with no requirement for human-curated domain-specific dictionaries. to this end, we rethink the definition and essence of "chinese words" and develop a word miner to obtain domain-specific words from unlabeled target domain data. moreover, a segmenter is trained on the source domain data to recognize the common words in unlabeled target data. this way, sentences from the target domain are assigned automatic annotations that can be used as target domain training data. although distant annotation could provide satisfactory labeled target domain data, there still exist annotation errors that affect the final performance. to reduce the effect of noisy data in automatic annotations in target domain data and make better use of source domain data, we propose to apply adversarial training jointly on the source domain dataset and the distantly constructed target domain dataset. and the adversarial training module can capture deeper domain-specific and domain-agnostic features. to show the effectiveness and robustness of our approach, we conduct extensive experiments on five real-world datasets across various domains. experimental results show that our approach achieves state-of-the-art results on all datasets, significantly outperforming representative previous works. further, we design sufficient subsidiary experiments to prove the alleviation of the aforementioned problems in cross-domain cws. ## our approach figure 2 shows the framework of our approach to cross-domain cws, which is mainly composed of two components: 1) distant annotation (da), and 2) adversarial training (at). in the following, we will describe details of the framework (daat) from the left to right in figure 2 . in this paper, bold-face letters (e.g. w ) are used to denote vectors, matrices and tensors. we use numerical subscripts to indicate the indices of a sequence or vector. we use the subscript of src to indicate the source domain and tgt to denote the target domain. ## experiments in this section, we conduct extensive cross-domain cws experiments on multiple real-world datasets with different domains, then comprehensively evaluate our method and other approaches. ## conclusion in this paper, we intuitively propose a unified framework via coupling distant annotation and adversarial training for the cross-domain cws task. in our method, we investigate an automatic distant annotator to build the labeled target domain dataset, effectively address the oov issue. further, an adversarial training procedure is designed to capture information from both the source and target domains. empirical results show that our framework significantly outperforms other proposed methods, achieving the state-of-the-art result on all five datasets across different domains.