# A Multi-Granularity Word Fusion Method for Chinese NER
## 1 Introduction  
Large-scaled pre-training language models, such as BERT [1] and RoBERTa [2], have become a fundamental backbone for various natural language processing (NLP) tasks. Due to the outstanding performance, a lot of work has applied these models to named entity recognition (NER). From the view of obtaining contextual semantics, for example, ERNIE [3] is designed to learn language representation that is strengthened by knowledge-masking techniques, including phrase- and entity-level masking. In order to direct word embedding learning, ERNIE learned intuitively about knowledge and longer semantic dependencies, such as the relationship between entities, the property of an entity, and the type of an event, as opposed to explicitly adding the knowledge embedding. As a result, the model is more generalizable and flexible. However, there are differences between Chinese and English, such as the way of splitting words, resulting in the NER model not being able to transform easily from English to Chinese. There has been a large amount of work focusing on Chinese NER. For example, by changing the word splitting method, BERT can be applied to Chinese NER. It is possibleto achieve a good result with careful execution. However, this approach suffers from the problem that BERT blocks part of the word-piece segmentation before training. To address this issue, [PERSON] et al. (2020) proposed BERT-wwm. Specifically, BERT-wwm introduces the whole word masking strategy for Chinese BERT. It masked the whole word with Chinese text, instead of masking the Chinese character. By doing this, the model shifts its emphasis from focusing solely on the characters to focusing more on the role that words play in a phrase when learning. We have a deeper perspective on properties of the Chinese language based on these works. We think there are two problems that hinder the Chinese NER from using the English NER model. The first problem is that multi-granularity word information is underutilized. In the Chinese language, the basic elements are characters and words, and they can form phrases. Characters, words, and variously sized phrases make up multi-granularity word information. Character and word information are usually used in Chinese NER models, but phrase information is often ignored. The length of phrases is not fixed, and might carry richer semantic information than the words and phrases. For example, the phrase \"Zhu Jian Bu\" is the \"Ministry of Housing and Urban-Rural Development\", which is an organization. \"Zhu Jian Bu\" consists of three characters, where \"Zhu\" means \"housing\", \"Jian\" means \"development\", and \"Bu\" means \"ministy\". Even though it's an abbreviation, we can easily understand what it means when we see it in the news. However, from the basic information of above three characters, it is difficult for models to recognize this as an organization, because the information they contain is insufficient. Humans associate frequently a number of words when they see characters and words, but the model does not. Based on this intuitive idea, we believe that the model will need to associate additional words, just like people. Therefore, in order to make the model obtain richer semantic information, we hope to introduce multi-granularity information into our model. The second problem is that word information could lead to word segmentation errors and noise. Using \"[PERSON] [PERSON]\"(\"Nanjing Yangtze River Bridge\" in English) as an example, the correct segmentation of it is \"[PERSON] / [PERSON] / [PERSON]\". However, it is wrong to segment the phrase into \"[PERSON] / [PERSON] / [PERSON]\", and the word segmentation could lead to the error propagation. Furthermore, for the character \"[PERSON]\", \"[PERSON]\" (\"Nanjing City\" in English) and \"[PERSON]\" (\"Mayor\" in English) both contain it, but their semantics are extremely different, resulting in noise. Word segmentation errors and noise could reduce the performance of models.