OPDAI at S em E val-2022 Task 11: A hybrid approach for C hinese NER using outside W ikipedia knowledge (2022)

Abstract: This article describes the OPDAI submission to SemEval-2022 Task 11 on Chinese complex NER. First, we explore the performance of model-based approaches and their ensemble, finding that fine-tuning the pre-trained Chinese RoBERTa-wwm model with word semantic representation and contextual gazetteer representation performs best among single models. However, the model-based approach performs poorly on test data because of low-context and unseen-entity cases. Then, we extend our system into two stages: (1) generating entity candidates by using neural model, soft-templates and Wikipedia lexicon. (2) predicting the final entity results within a feature-based rank model. For the evaluation, our best submission achieves an F_1 score of 0.7954 and attains the third-best score in the Chinese sub-track.

## introduction named entity recognition (ner) @xcite aims to detecting the boundaries of named entities and recognizing their categories(e.g., person or location). it plays an important role in many downstream tasks, such as information extraction and question answering. semeval-2022 task 11: multilingual complex named entity recognition(multiconer) @xcite ) is a shared task which encourages participants to develop ner system to detect semantically ambiguous and complex entities in short and low-context settings for 11 languages. participants can build a ner model that works only for one language or for all the languages. and an additional track with code-mixed data are offered in this task. different from ordinary ner, this task focuses on complex and unseen entities. complex entities, like the titles of creative works(movie/book/software names) are harder to recognize. additional test sets on questions and short search queries are offered in the test phase, which contains large proportion of unseen entities. our main interest is to build a ner system which can process complex entities and adapt to other domains in practical scenarios in chinese language. this paper describes our two-stage hybrid approach for chinese ner. a description of datasets provided in this shared task and additional datasets adopted in our system is given in section 3. the implementation details of our system are listed in section 4. we first experiment with model-based methods and integrate word semantic feature and gazetteer feature with neural model to improve inference performance. further, we extend our model system to a two-stage prediction system: entity candidates generation and entity confidence ranking. confidence ranking is used to pick out highconfidence entities from candidates. with the help of wikipedia lexicon and soft templates for generating entity candidates, our hybrid system shows good performance and great domain adaption capability in the final evaluation phase. ## related work our work is mainly related to the pre-trained language models and some specific strategies for chinese ner task. ## data we experiment using the datasets shared by mul-ticoner @xcite on chinese monolingual track, which consist of 15300 training sentences, 800 validation sentences and 151661 test sentences. entities are labeled using bio scheme, and six entity types are involved: person(per), location(loc), group(grp), corporation(corp), product(prod) and creative work(cw). in order to make our model better adapt to ambiguous semantics and insufficient context, we built an entity lexicon from wikipedia data. we parsed a wikidata dump and mapped all the entities to our ner taxonomy following the rule from ## methodology our system classifies ner task into two stages: entity candidates generation and entity confidence ranking. based on bert-based model ensembles, soft-template methods and wikipeida lexicon, we can first generate entity candidates. and then, handcrafted features for each entity candidate are extracted. finally, a machine learning based rank model is used for entity confidence rank, the entities whose confidence score is above the threshold are regarded as the final predictions. figure 1 gives an overview of our hybrid approach. 4.1 entity candidates generation three approaches are adopted for entity candidates generation: model-based, template-based and lexicon-based. bert-based model ensembles can achieve named entities recognition, however, its recall performance significantly decreases when dealing with low-context or new-entity cases. to alleviate this effect, soft templates and wikipedia lexicon is integrated for candidates recall. pre-trained model encoding aggregation inference [cls] tok1 tok2 tok3 ... ... ... [sep] pad pad [cls] here, el represents the wikipedia entity lexicon. w i,k stands for the span that begins with c i and ends with c k . the average word embeddings of each word set are concatenated as a 600-dimension vector(200 for each word set) is regarded as the final word semantic representation. contextual gazetteer representation for each character is a 13-dimension binary vector, which is introduced by @xcite . ## conclusion in this paper, we introduce a hybrid approach for chinese ner, which contains two stages: entity candidates generation and confidence ranking. we find that integrating word semantic representation and gazetteer representation can improve the performance of neural model-based approach. in particular, our ensemble model of different aggregation strategies performs better. however, due to the limitation of training data, the performance of neural model-based approach drops sharply in other new domains. considering that there are new entities and other domain sentences in test sets, we can use wikipeida lexicon and soft templates to help recall unseen entities, and an entity confidence ranking model is built which results in significant improvement on test sets. for future work, semi-supervised approaches or data augmentation methods could be explored to alleviate the limitation of training data.