Incorporating Uncertain Segmentation Information into C hinese NER for Social Media Text (2020)

Abstract: Chinese word segmentation is necessary to provide word-level information for Chinese named entity recognition (NER) systems. However, segmentation error propagation is a challenge for Chinese NER while processing colloquial data like social media text. In this paper, we propose a model (UIcwsNN) that specializes in identifying entities from Chinese social media text, especially by leveraging uncertain information of word segmentation. Such ambiguous information contains all the potential segmentation states of a sentence that provides a channel for the model to infer deep word-level characteristics. We propose a trilogy (i.e., Candidate Position Embedding => Position Selective Attention => Adaptive Word Convolution) to encode uncertain word segmentation information and acquire appropriate word-level representation. Experimental results on the social media corpus show that our model alleviates the segmentation error cascading trouble effectively, and achieves a significant performance improvement of 2% over previous state-of-the-art methods.

## introduction named entity recognition (ner) is a fundamental task for natural language processing and fulfills lots of downstream applications, such as semantic understanding of social media contents. chinese ner is often considered as a characterwise sequence labeling task since there are no natural delimiters between chinese words @xcite . but the word-level information is necessary for a chinese ner system @xcite @xcite . various segmentation features can be obtained from the chinese word segmentation (cws) procedures then used into a pipeline ner module @xcite @xcite , or be co-trained by an interesting instance "南京市长江大桥调研(daqiao jiang, major of nanjing city, is investigating)..." is represented, which is cited from @xcite . cws-ner multi-task learning @xcite . however, segmentation error propagation is a challenge for chinese ner, when processing informal data like social media text @xcite . the cws will produce more unreliable results on the social media text than on the formal data. incorrectly segmented entity boundaries may lead to ner errors. nevertheless, most existing extractors always assume that input segmentation information is affirmative and reliable without conscious error discrimination. that is, they acquiesce in that "the one supposed-reliable word segmentation output of a cws module will be input into the ner module". although the joint training way may improve the accuracy of word segmentations, the ner module still cannot recognize inevitable segmentation errors. to solve this problem, we design a model (uicwsnn) that dedicates to identifying entities from chinese social media text, by incorporating uncertain information of chinese word segmentation into a neural network. this kind of uncertain information reflects all the potential segmentation states of a sentence, not just the certain one that is supposed-reliable by the cws module. furthermore, we propose a trilogy to encode uncertain word segmentation information and acquire word-level representation, as shown in figure 1 . in summary, the contributions of this paper are as follows: the ner on english has achieved promising performance by naturally integrating character information into word representations @xcite @xcite @xcite . however, chinese ner is still underachieving because of the word segmentation problem. unlike the english language, words in chinese sentences are not separated by spaces, so that we cannot get chinese words without pre-processed cws. in particular, identifying entities on chinese social media is harder than on other formal text since there is worse segmentation error propagation trouble. existing methods payed little attention to this issue, and there were few entity recognition methods specifically for chinese social media text @xcite . as for the chinese ner, existing methods could be classified as either word-wise or character-wise. the former one used words as the basic tagging unit @xcite . segmentation errors would be directly and inevitably entered into ner systems. the latter used characters as the basic tokens in the tagging process @xcite @xcite . character-wise methods that outperformed wordwise methods for chinese ner @xcite . there were two main ways to take word-level information into a character-wise model. one was to employ various segmentation information as feature vectors into a cascaded ner model. chinese word segmentation was performed first before applying character sequence labeling @xcite @xcite . the pre-processing segmentation features included character positional embedding @xcite , segmentation tags @xcite , word embedding @xcite @xcite and so on. the other was to train ner and cws tasks jointly to incorporate task-shared word boundary information from the cws into the ner @xcite @xcite . although co-training might improve the validity of the word segmentation, the ner module still had no specific measures to avoid segmentation errors. the above existing methods suffered the potential issue of error propagation. a few researchers tried to address the above defect. @xcite used multiple word segmentation outputs as additional features to a ner model. however, they treated the segmentations equally without error discrimination. @xcite introduced four naive selection strategies to select words from the pre-prepared lexicon for their model. however, these strategies did not consider the context of a sentence. @xcite proposed a lattice lstm model that used the gated recurrent units to control the contribution of the potential words. however, as shown by @xcite , the gate mechanism might cause the model to degenerate into a partial word-based model. @xcite and @xcite proposed the models with graph neural network based on the information that the gazetteers or lexicons offered. obtaining largescale, high-quality lexicons would be costly. they were dedicated to capturing the correct segmentation information but might not alleviate the interference of inappropriate segmentations. it is worth mentioning that the above methods were not specifically aimed at social media. we propose a method to learn word-level representation by leveraging uncertain word segmentation information while considering the informal expression characteristics of social media text. ## methodology figure 1 illustrates the overall architecture of our model uicwsnn. given a sentence @xmath0, c 2 , • • • , c n } as the sequence of characters, each character will be assigned a pre-prepared tag. we use a conditional random fields (crf) layer to decode tags according to the outputs from the sequence encoder @xcite . as for the sequence encoding, we use the convolution operation as our basic encoding unit. the colloquial social media text usually does not have normative grammar or syntax and presents semantics in fragmented form, for example, "有好多好 多的话想对你说李巾凡想要瘦瘦瘦成李帆我 是想切开云朵的心(have many many words to say to you jinfan li wanna thin thin thin to fan li i am a heart that want to cut the cloud)". these properties will destroy the propagation of temporal semantic information that comes with the textual sequence. therefore, the convolutional neural network (cnn) is naturally suitable for encoding colloquial text because it specializes in capturing salient local features from a sequence. more importantly, we use a trilogy to learn the word-level representation by incorporating uncertain information of chinese text segmentation, as shown in the following details. ## conclusion named entity recognization is an urgent task for semantic understanding of social media content. as for the chinese ner, chinese word segmentation error propagation is prominent since there is much colloquial text in social media. in this paper, we explore a trilogy to leverage the uncertain information of word segmentation to avoid the interference of segmentation errors. the step-1 utilizes the candidate position embedding to present the potential segmentation states of a sentence; the step-2 employs the position selective attention to capture appropriate segmentation states while ignoring unreliable parts; the step-3 uses the adaptive word convolution to encode word-level representation dynamically. we analyze the performance of each component of the model and discuss the relationship between the model and related factors such as segmentation error, bert, and entity length. experiment results on different datasets