# Prompt-Based Word-Level Information Injection BERT for Chinese Named Entity Recognition
## 1 Introduction  
Named entity recognition (NER) tries to distinguish the entities' boundary and category labels from unstructured text, which is an important natural language processing task, owing to the application in several domains of information extraction, question answering, machine translation, and so on. NER was formulated as a sequence labeling problem in earlier research, and several of the current works for NER are established based on neural network approaches [1; 2; 3; 4; 5; 6; 7; 8; 9]. However, Chinese NER is correlated with word segmentation. In detail, entities' boundaries and segmentation boundaries tend to overlap each other. Therefore, some of the pipeline methods use an intuitive way to segment the sentence before approaching the Chinese NER task. Similar to other pipeline methods, the method mentioned above faces the problem of error propagation. From another aspect, name entities are a considerable solution for out-of-vocabulary segmentation, which may further exacerbate entity recognition errors. In order to enhance the character-based Chinese NER methods, two groups of models introduce the segmentation information. One of them explicitly models the word information by accelerating the additional word or other relevant features' embedding with the character features [10; 11]. As their validity has been verified, pre-trained language embeddings, such as BERT [12], aim to inject the implicit word-level syntactic and semanticknowledge. The other group modifies the original encoding structure to accommodate the situation where character and word information are both used as input [13; 14; 15; 16; 17]. Although some of the recent work [18; 19] considers the combination of these two lines of research, the methods do not utilize the syntactic and semantic knowledge extracted by original language model representation in the process of utilizing BERT. On the other side, the category information is not efficiently capitalized. Illustrated in Figure 1, the original text is used to construct several word-character lattices according to the pre-trained word dictionary and their word embedding. As a result, the words inside the original sequence are \"(Nanjing)\", \"(Mayor)\", \"(Yangtze River Bridge)\", \"(Nanjing City)\", \"(Daqiao Jiang)\", \"(Bridge)\", and \"(Yangtze)\". For the token \"(Daqiao Jiang)\", we need to consider the category information to distinguish whether the word is a named entity. While only considering the shallow word segmentation information, the entity may be recognized as \"(Bridge)\". Inspired by the work about BERT Adapter [20; 21; 22] and the prompt learning, we propose Prompt based Word-level Information Injection BERT (PWII-BERT) to integrate prompt guided word features inside the pre-trained language model immediately. By preparing the domain-related dictionary in advance, the raw sentence is transformed into a sequence of characters with their paired corresponding words. Specifically, we engineer a Word-level Information Injection Adapter (WIIA) for dynamically fusing the most relevant word-level feature for all the characters through a biaffine mechanism. In addition, we automatically extract the soft prompt for guiding the word-level information injection. The WIIA is inserted between the Transformer encoder and prompt-guided Transformer layers. By using original pre-trained language models, prompt-based word-level information is integrated through fine-tuning both the language model and extra parameters, which is dissimilar to the classical Adapter fixing the BERT parameters. The effectuality of PWII-BERT is verified on four widely used Chinese NER datasets. One of the main advantages of PWII-BERT is its ability to incorporate both category information and lexicon features, which authorize it to catch both syntactic and semantic information for Chinese NER and achieve further improvements compared with several state-of-the-art methods.