CL a C at S em E val-2023 Task 2: Comparing Span-Prediction and Sequence-Labeling Approaches for NER (2023)

Abstract: This paper summarizes the CLaC submission for the MultiCoNER 2 task which concerns the recognition of complex, fine-grained named entities. We compare two popular approaches for NER, namely SequenceLabeling and Span Prediction. We find that our best Span Prediction system performs slightly better than our best Sequence Labeling system on test data. Moreover, we find that using the larger version of XLM RoBERTa significantly improves performance. Post-competition experiments show that Span Prediction and Sequence Labeling approaches improve when they use special input tokens ([s] and [/s]) of XLM-RoBERTa. The code for training all models, preprocessing, and post-processing is available at https://github.com/harshshredding/semeval2023-multiconer-paper .

## introduction multiconer 2 @xcite is the second version of the multiconer @xcite shared task. it focuses on identifying complex named entities in short sentences that lack context and addresses the challenge of identifying and distinguishing 30 fine-grained entity types and of handling simulated errors, such as typos. competition data is tokenzied and tagged with the bio scheme, i.e. the label b for begin, i for inside, and o for outside target entities. for this challenge, the 30 entity types were distinguished using suffixes to the b and i labels, e.g. b-corp and i-corp, as illustrated in example 1. ner is frequently formulated as a sequence labeling problem @xcite @xcite , in which a model learns to label each token individually using the bio labeling scheme. in contrast, span prediction approaches @xcite @xcite label entire text spans with entity types. recently, @xcite compared the two approaches and found that sequence labeling performed better for long entities with low label consistency but that span prediction performed better for out of vocabulary words and entities of short to medium length. because multi-coner 2 tests both, the ability to identify out of vocabulary entities and entities of varying length foot_0 , it is interesting to compare the two approaches on this task. our submission to the english track does not use any external knowledge and holds rank 21 of 41 on codalab foot_1 with a macro-f1 of 59. ## results and discussion we evaluate all of our systems on the codalab competition website 3 , where submissions are evaluated using the entity level macro f1 metric. all models use the xlm-roberta large lm unless otherwise indicated. post-competition experimentation showed that increasing the batch size from 1 to 4 improved results by at least 3.2, this improvement is shown in we also found that adding the embeddings for roberta special @xmath0, especially raising performance of the sequence model seq by 2.1, nearly erasing the performance advantage of the spanbased model. the results are shown in the span prediction models showed significantly higher performance on the validation set than the test set. the post-competition improvements affect the sequence and span-based models differently and nearly erase the performance difference.