Simplify the Usage of Lexicon in C hinese NER (2020)

Abstract: Recently, many works have tried to augment the performance of Chinese named entity recognition (NER) using word lexicons. As a representative, Lattice-LSTM has achieved new benchmark results on several public Chinese NER datasets. However, Lattice-LSTM has a complex model architecture. This limits its application in many industrial areas where real-time NER responses are needed. In this work, we propose a simple but effective method for incorporating the word lexicon into the character representations. This method avoids designing a complicated sequence modeling architecture, and for any neural NER model, it requires only subtle adjustment of the character representation layer to introduce the lexicon information. Experimental studies on four benchmark Chinese NER datasets show that our method achieves an inference speed up to 6.15 times faster than those of state-of-the-art methods, along with a better performance. The experimental results also show that the proposed method can be easily incorporated with pre-trained models like BERT.

## introduction named entity recognition (ner) is concerned with the identification of named entities, such as persons, locations, and organizations, in unstructured text. ner plays an important role in many downstream tasks, including knowledge base construction @xcite , information retrieval @xcite , and question answering @xcite . in languages where words are naturally separated (e.g., english), ner has been conventionally formulated as a sequence labeling problem, and the state-of-the-art results have been achieved using neural-network-based models @xcite @xcite . compared with ner in english, chinese ner is more difficult since sentences in chinese are not naturally segmented. thus, a common practice for chinese ner is to first perform word segmentation using an existing cws system and then apply a word-level sequence labeling model to the segmented sentence @xcite . however, it is inevitable that the cws system will incorrectly segment query sentences. this will result in errors in the detection of entity boundary and the prediction of entity category in ner. therefore, some approaches resort to performing chinese ner directly at the character level, which has been empirically proven to be effective @xcite @xcite @xcite @xcite . a drawback of the purely character-based ner method is that the word information is not fully exploited. with this consideration, @xcite proposed lattice-lstm for incorporating word lexicons into the character-based ner model. moreover, rather than heuristically choosing a word for the character when it matches multiple words in the lexicon, the authors proposed to preserve all words that match the character, leaving the subsequent ner model to determine which word to apply. to realize this idea, they introduced an elaborate modification to the sequence modeling layer of the lstm-crf model @xcite . experimental studies on four chinese ner datasets have verified the effectiveness of lattice-lstm. however, the model architecture of lattice-lstm is quite complicated. in order to introduce lexicon information, lattice-lstm adds several additional edges between nonadjacent characters in the input sequence, which significantly slows its training and inference speeds. in addition, it is difficult to transfer the structure of lattice-lstm to other neural-network architectures (e.g., convolutional neural networks and transformers) that may be more suitable for some specific tasks. in this work, we propose a simpler method to realize the idea of lattice-lstm, i.e., incorporating all the matched words for each character to a character-based ner model. the first principle of our model design is to achieve a fast inference speed. to this end, we propose to encode lexicon information in the character representations, and we design the encoding scheme to preserve as much of the lexicon matching results as possible. compared with lattice-lstm, our method avoids the need for a complicated model architecture, is easier to implement, and can be quickly adapted to any appropriate neural ner model by adjusting the character representation layer. in addition, ablation studies show the superiority of our method in incorporating more complete and distinct lexicon information, as well as introducing a more effective word-weighting strategy. the contributions of this work can be summarized as follows: we performed experiments on four public chinese ner datasets. the experimental results show that when implementing the sequence modeling layer with a single-layer bi-lstm, our method achieves considerable improvements over the state-of-theart methods in both inference speed and sequence labeling performance. ## background in this section, we introduce several previous works that influenced our work, including the softword technique and lattice-lstm. ## approach in this work, we sought to retain the merits of lattice-lstm while overcoming its drawbacks. to this end, we propose a novel method in which lexicon information is introduced by simply adjusting the character representation layer of an ner model. we refer to this method as softlexicon. as shown in figure 1 , the overall architecture of the proposed method is as follows. first, each character of the input sequence is mapped into a dense vector. next, the softlexicon feature is constructed and added to the representation of each character. then, these augmented character representations are put into the sequence modeling layer and the crf layer to obtain the final predictions. ## conclusion in this work, we addressed the computational efficiency of utilizing word lexicons in chinese ner. to obtain a high-performing chinese ner system with a fast inference speed, we proposed a novel method to incorporate the lexicon information into the character representations. experimental studies on four benchmark chinese ner datasets reveal that our method can achieve a much faster inference speed and better performance than the compared state-of-the-art methods.