Janko at S em E val-2023 Task 2: Bidirectional LSTM Model Based on Pre-training for C hinese Named Entity Recognition (2023)

Abstract: This paper describes the method we submitted as the Janko team in the SemEval-2023 Task 2,Multilingual Complex Named Entity Recognition (MultiCoNER 2). We only participated in the Chinese track. In this paper, we implement the BERT-BiLSTM-RDrop model. We use the fine-tuned BERT models, take the output of BERT as the input of the BiLSTM network, and finally use R-Drop technology to optimize the loss function. Our submission achieved a macro-averaged F1 score of 0.579 on the testset.

## introduction named entity recognition (ner) is to identify entities with specific meanings from a given text. named entities generally refer to entities with a specific meaning or strong referential meaning in the text, usually including personal names, place names, organizational structure names, etc. ner is a key task in natural language processing, and it is very useful in nlp tasks such as information extraction @xcite , question answering system @xcite , and machine translation @xcite . however, due to the nature of the language, ner is also a challenging problem. first, a word often has multiple meanings, leading to different categories of entities in different sentences. for example, washington can mean either a place name or a person's name. secondly, the boundary of the named entity is vague, and its forms are various. it can be a simple noun, a gerund phrase, or even a complete sentence. multiconer 2 @xcite aims to identify named entities with finer precision, and the organizers divided the whole competition into 13 tracks based on language. each data set @xcite is composed of a training set, a validation set, and a test set. the training and validation set provides labels for named entities, and the test set provides text only. compared with multiconer @xcite , the number of entity tags has been increased from 6 @xcite to more than 30, while the semantics are vaguer and the text length is shorter. in addition, mock errors are added to the test set to make the task more realistic and difficult. similar to other ner tasks, multiconer 2 uses b-i-o tags to label the tokens, where "b" indicates the first token of a concept, "i" indicates tokens inside of a concept, and "o" indicates tokens that don't belong to any entity. this paper explains our submission to the chinese track in multiconer 2. we implemented a simple neural network system based on a pretrained language model. our submission ranked 14th among 21 teams with 0.579 f1 score in the test set. this paper is organized as follows: in section 2, we will briefly introduce the research status of the ner task in chinese. section 3 describes the models and methods used. section 4 introduces the implementation details and experimental results. finally, this paper is summarized in section 5. ## related work english ner task is based on characters, but the chinese ner task is more difficult than the english ner task because the meaning of words is much larger than a single word and chinese has no space and the basic unit is the word @xcite . the development of ner in chinese can be roughly divided into three categories: rule-based approach, traditional machine learning approach, and deep learning approach. ## methodology in this section, we will introduce the system architecture we use in the chinese ner task. the system will be composed of three parts. the model we will eventually use to predict the test set files is called bert-bilstm-rdrop.the architecture of the bert-bilstm-rdrop model is shown in figure 1 . ## conclusion in this paper, we described our method, which is based on the pre-trained bert model and is used to predict the ner task semeval 2023 task 2: multilingual complex named entity recognition. we implemented the bert-bilstm-rdrop model and prove the validity of the different model compositions. our submission achieved a macro-averaged f1 score of 0.579 on the test set. in the future, we will continue to improve the system, use more advanced models, and try model integration methods.