Advancing Multi-Criteria C hinese Word Segmentation Through Criterion Classification and Denoising (2023)

Abstract: Recent research on multi-criteria Chinese word segmentation (MCCWS) mainly focuses on building complex private structures, adding more handcrafted features, or introducing complex optimization processes. In this work, we show that through a simple yet elegant input-hint-based MCCWS model, we can achieve state-of-the-art (SoTA) performances on several datasets simultaneously. We further propose a novel criterion-denoising objective that hurts slightly on F1 score but achieves SoTA recall on out-of-vocabulary words. Our result establishes a simple yet strong baseline for future MCCWS research. Source code is available at https://github.com/IKMLab/MCCWS .

## introduction chinese word segmentation (cws) is a preliminary step for performing chinese nlp tasks. researchers have proposed many cws datasets to enhance word segmentation performance in different text domains. however, due to the divergence in linguistic perspectives, the same text passage can be segmented in entirely different ways across datasets. for example, in their written forms, chinese human names have no spaces in between. some datasets segment human names into last and first names, while others leave human names as a whole (see ## related works after @xcite proposed to treat cws as a character tagging problem, many works followed the same problem formulation to address cws. @xcite is the first to propose a multi-criteria learning framework for cws. they proposed multiple private-structure-based mccws models and trained them in an adversarial setting. a criterion discriminator was used in their adversarial training so that common knowledge across datasets could be shared through different private structures. but the nature of adversarial training forces their criterion discriminator to predict each criterion with equal probability @xcite . thus their criterion discriminator failed to provide accurate criterion prediction and cannot be used to choose a suitable criterion for each input. inspired by the success of the bilstm-based sccws model @xcite and input-hintbased multilingual neural machine translation system @xcite , @xcite proposed to build an input-hint-based mccws on top of the bilstm. they added two artificial tokens representing a criterion and put them at the beginning and the end of an input sentence. such a simple idea advanced the sota performance on seven datasets simultaneously. @xcite proposed switch-lstms, which can dynamically route between multiple bilstms to encode criterionspecific features when given different input hints. their work set the sota limit that can be achieved via lstm architecture. after the remarkable effectiveness of pre-trained language models was found, mccws works started to replace lstm encoders with transformer encoders @xcite . @xcite used roberta @xcite to build an input-hint-based mccws model, which advanced sota performance. @xcite shows that adding private structures on top of a large pre-trained model can push sota even further. @xcite pre-trained an input-hintbased mccws on bert @xcite with meta-learning @xcite , but only after finetuning did they become the new sota on sccws models. @xcite and @xcite are the most similar to ours among many mccws works. we use a nearly identical input-hint-based model as in @xcite . however, like all the works mentioned before, they do not include a criterion classification objective, and therefore fail to provide a way to choose criteria automatically. @xcite is the only work using criterion classification objective, but we further simplified its model structure, which outperforms their models on average f1-score. we further proposed a novel criterion-denoising objective that helps choose criteria automatically. by trading off 0.07% f1-score on average, we achieved the new sota on the oov recall, which improved by a large margin compared to the previous sota (1.61%). in summary, previous research on mccws either did not provide a way to choose a criterion or always manually chose a criterion. in our work, we proposed a simple yet elegant way to make our mccws model automatically choose a suitable criterion for the given character sequence. comparing our works to others, we find that (1) our model has the simplest structure and is the easiest to implement among other works; (2) we achieved mc-cws sota performance on several cws datasets and on average f1-score over 10 datasets; (3) we improved sota oov recall by a large margin. ## mccws in this section, we describe the detail of our methodology. we first give a formal definition of inputhint-based mccws (section 3.1). then we introduce our mccws model (section 3.2). finally, we formally define our criterion-denoising objective and describe how to jointly train our mccws on top of the proposed denoising objective (section 3.3).