CMB AI Lab at S em E val-2022 Task 11: A Two-Stage Approach for Complex Named Entity Recognition via Span Boundary Detection and Span Classification (2022)

Abstract: This paper presents a solution for the SemEval-2022 Task 11 Multilingual Complex Named Entity Recognition. What is challenging in this task is detecting semantically ambiguous and complex entities in short and low-context settings. Our team (CMB AI Lab) propose a two-stage method to recognize the named entities: first, a model based on biaffine layer is built to predict span boundaries, and then a span classification model based on pooling layer is built to predict semantic tags of the spans. The basic pre-trained models we choose are XLM-RoBERTa and mT5. The evaluation result of our approach achieves an F1 score of 84.62 on sub-task 13, which ranks the third on the learder board.

## introduction named entity recognition (ner) @xcite is a fundamental task in natural language processing, aiming at identifying the spans of texts that refer to entities. ner is widely applied to information extraction and data mining @xcite , which is greatly challenging in practical and open domain settings. however, the previous research has not paid much attention on processing complex and ambiguous named entities. semeval 2022 task 11 @xcite ) containing a total of 13 sub-tasks is a complex ner task which focuses on detecting semantically ambiguous and complex entities in short and lowcontext settings @xcite . for the purpose of testing the domain adaption capability of the participating models, the task not only set 11 base sub-tasks: english, spanish, dutch, russian, turkish, korean, farsi, german, chinese, hindi and bangla, but also set two additional testing sets on questions and short queries: multilingual, and code-mixed @xcite . we conduct a two-stage method to deal with the code-mixed subtask, which achieves an f1 score of 84.62. this paper is structured as follows. the related work of ner is briefly introduced in section 2. the data for training and testing the model is presented in section 3. the details of the two-stage method is described in section 4. the experimental results of our method are exhibited in section 5. section 6 summarizes this paper. ## related work in the nlp field, the ner task is usually considered as a sequence labeling problem @xcite @xcite . with well-designed features, crf-based models have achieved the leading performance @xcite @xcite . recently, neural network models have been exploited for feature representations @xcite . moreover, contextualized word representations such as elmo @xcite and bert @xcite have also achieved great success. as for ner, the end-to-end bidirectional lstm + crf model @xcite is one representative architecture. these models are only capable of recognizing regular named entities. in e-commerce search domain, a common scenario is code-mixed queries, with query terms composed of multiple languages @xcite . the application for code-mixed web queries still remains challenging @xcite . a recent work proposed an ner hybrid approach for codemixed queries, consisting of a gazetteer and tree based identifier @xcite . another work leverages linguistic features to train a conditional random field (crf) model, where the output is further processed using multi-lingual gazetteer lists @xcite . in seme val 2022 task 11 code-mixed sub-task, we use a two-stage ner approach and get the 3rd place in the competition. ## data in this task, we use the official raw data @xcite to train and test our model. each line of texts in the data belongs to a sample, the languages involved are: english, spanish, dutch, russian, turkish, korean, farsi, german, chinese, hindi, and bangla, some of which are also provided with code-mixed data as an additional subtask. entity types include person, location, group, corporation, product, and creative work. the participants have to use their systems to accurately detect the entities and submit the predictions for the mixed languages task. in a data file, samples are separated by blank lines. each data instance is tokenized and each line contains a single token in the first column with the associated label in the last (4th) column. the second and third columns are underscores (_) to separate the tokens and the labels. the entities are labeled with the bio scheme, which means that the token tagged o is not a part of the entity, the token tagged b-x is the first token of an x type entity, and the remaining tokens of the entity are tagged as i-x. when the amount of training data is insufficient or unevenly distributed, data augmentation can quickly expand the corpus to avoid overfitting. at the same time, data augmentation can also improve the robustness of the model, preventing the performance of the model from being greatly reduced once the data only changes slightly. we build a dictionary with all entities of the same type to randomly replace the entities in each sample, and translate the replaced entities into other languages to expand the dataset, which is similar is to autoencoders in the computer vision. however, translation between different languages relies on a large number of parallel corpuses, and requires training first. ## methodology the two-stage method we use in this task includes two separated models to recognize the named entities: one for predicting the boundaries of the spans, and the other for predicting the semantic tags of the spans. the processing flow of our approach is depicted in figure 1 . ## results a two-stage method is employed to complete the code-mixed language sub-task. based on two pretrained models (xlm-roberta and mt5), we adopted a variety of optimization schemes, such as: biaffine network structure, two-stage entity prediction, adding distantly supervised dictionary and adversarial training, all of which have achieved a certain improvement, according to the evaluation results shown in with the xlm-roberta + crf method as the baseline, and an end-to-end structure, we get an f1 score of 79.7. after using the biaffine network structure and two-stage optimization architecture instead, the f1 score improves to 80.9 and 81.3 respectively. in addition, the two-stage optimization architecture introducing supervised dictionary, adversarial training and data augmentation obtains f1 scores of 82.5, 83.1 and 82.7 respectively. compared to the xlm-roberta, the prediction results acquired based on the mt5 pre-trained model are improved by an average of 0.4 points. as a result, the final evaluation scores gained by voting is 84.62. ## conclusion aiming at the complex multilingual ambiguity and lack of context in this competition, we adopt a deep learning network model for entity extraction based on the biaffine attention mechanism, and carry out transfer learning based on different pretrained models such as roberta and mt5. through adversarial training, the robustness of the model is enhanced, and the two-stage training also improves the performance of the model in fewshot scenarios. besides, a remote supervised dictionary is added to revise the results, and the entity dictionary for random replacement and multilingual machine translation is used for data augmentation. usually for enhanced data, it is necessary to give a weight less than 1, which is different from