CWS eg: An Efficient and General Approach to C hinese Word Segmentation (2023)

Abstract: In this work, we report our efforts in advancing Chinese Word Segmentation for the purpose of rapid deployment in different applications. The pre-trained language model (PLM) based segmentation methods have achieved state-of-the-art (SOTA) performance, whereas this paradigm also poses challenges in the deployment. It includes the balance between performance and cost, segmentation ambiguity due to domain diversity and vague words boundary, and multi-grained segmentation. In this context, we propose a simple yet effective approach, namely CWSeg, to augment PLM-based schemes by developing cohort training and versatile decoding strategies. Extensive experiments on benchmark datasets demonstrate the efficiency and generalization of our approach. The corresponding segmentation system is also implemented for practical usage and the demo is recorded.

## introduction chinese word segmentation (cws) is a preliminary but essential procedure for chinese language processing tasks, and has been applied in various scenarios @xcite @xcite @xcite @xcite . especially for fast complete recall and accurate semantic understanding in search and recommendation scenarios @xcite , cws is still indispensable. in addition, experiments on chinese llama and alpaca show that the token throughput of the model that expands the vocabulary through word segmentation has greatly improved the processing of chinese text compared with the original model @xcite . recent deep learning methods have achieved remarkable results on publicly available datasets in this regard @xcite . also, the pre-trained language model (plm) @xcite ) further + work was done at sensetime research * corresponding author emerges as the paramount foundation of text representation for cws as seen in other tasks @xcite @xcite . current plm-based approaches, however, pose three hurdles to the production deployment we need to cross: (1) one dilemma is the trade-off between the model performance and inference speed. (2) the lexical diversity and domain gap also jeopardize the fast deployment of a generic model to customized scenarios. @xcite . (3) plm-based schemes with single granularity are less likely to meet multi-granularity demands of practical relevance. to tackle these issues, we propose an efficient and general approach to augmenting plm-based chinese word segmentation methods, namely cwseg. it can extrapolate to different sequence labeling scenarios. recent studies showed that small models also have the potential to be comparable to large models @xcite . we thus introduce a new cohort training strategy to co-train a cohort of multi-scale model artifacts to meet the performance and real-time demands. specifically, we employ wasserstein distance (wd) @xcite to orchestrate distributions of model cohorts to enable more robust learning. in addition, we propose to construct the tailored domain-specific lexicon trie @xcite and build up a versatile decoding scheme to augment the optimal segmentation path searching on the fly for diverse practical scenarios. it can flexibly adjust the segmentation granularity and benefit customized domains. in summary, our primary goal is to build a versatile framework for strengthening different models simultaneously and then rapidly deploying them into multiple practical scenarios of cws, which is fundamentally different from existing research works. essentially, the output models of this framework can be regarded as complements to, not re-placements for, existing sota methods. experimental results on multiple benchmark datasets demonstrate the effectiveness of our approach. ablation studies confirm the necessity of cohort training strategy and lexicon trie aided versatile decoding solution. the cross-domain application experiments demonstrate the generalization capacity of our holistic approach. ## related work early work in chinese word segmentation builds upon the statistical assumption @xcite ) by modeling rules into the learning process. recently, plms have been introduced @xcite and made significant advances in this regard. our work, however, aims to alleviate their potential challenges involved in the industrial applications as mentioned in section 1. recent works @xcite distill knowledge from the well-trained teacher model into a student model to balance the model scale and performance. however, it requires multiple finetuning rounds and models can't learn from each other collaboratively. in this work, we introduce a cohort training based learning strategy to address these two problems for cws. different from the pioneering mutual learning @xcite in computer vision, we propose wasserstein distance to better enable the learning as studied in sec. 4.3. it's a more carbon-footprint-friendly solution as compared to recent research threads. to mitigate the effects of chinese lexical diversity, qiu et al. @xcite proposed a concise unified model to extract the criterion-aware representation for multi-criteria corpus, which requires training from scratch on the entire corpus for new criteria or domains. gong et al. @xcite ) proposed a multi-grained word segmentation by training with large-scale pseudo labels, which is relatively lagging for rapid deployment to new domains. our work approaches this issue by a lightweight versatile decoding scheme to sidestep heavy training loads. ## methodology as shown in fig. 1 (a) , we formulate cws as a classical sequence labeling problem as with existing compelling schemes. concretely, given a text sequence of n characters @xmath0, . . . , x n }, cws is to tag involved characters sequentially with the bio encoding by maximizing their joint probability p(y 1 , . . . , y n |x ) where y @xmath1, i, o}, short for beginning, inside and outside respectively. ## discussion our latency comparisons are benchmarked on the same cpu device, while deep methods run faster on cuda devices. besides, we can resort to a fastcompiling language (e.g., c++) backed platform or tailored toolchain (e.g., onnx) to optimize the serving speed. how to apply diversity modes to different scenarios? generally speaking, the coarse mode is to perceive complete semantics, and the fine mode is to perceive more extensive concepts. for example, in the scenarios of search and recommendation, the normal or coarse mode is employed to process web pages to build inverted indexes. index mode is often used for query expansion, where we disassemble queries into multiple granularities to maximize recall of relevant documents. ## conclusion in this work, we develop an efficient and general framework, cwseg, which enables the state-of-theart schemes of chinese word segmentation better prepared for industrial deployment scenarios. we present wasserstein distance-based cohort learning method and versatile decoding to facilitate the trade-off between segmentation performance and serving latency as well as the fast cross-domain adaption. comprehensive experiments are performed to justify the efficiency and generalization of cwseg. we believe that our work can be extrapolated to other sequence labeling problems straightforwardly.