Split- NER : Named Entity Recognition via Two Question-Answering-based Classifications (2023)

Abstract: In this work, we address the NER problem by splitting it into two logical sub-tasks: (1) Span Detection which simply extracts entity mention spans irrespective of entity type; (2) Span Classification which classifies the spans into their entity types. Further, we formulate both sub-tasks as question-answering (QA) problems and produce two leaner models which can be optimized separately for each sub-task. Experiments with four cross-domain datasets demonstrate that this two-step approach is both effective and time efficient. Our system, SplitNER outperforms baselines on OntoNotes5.0, WNUT17 and a cybersecurity dataset and gives on-par performance on BioNLP13CG. In all cases, it achieves a significant reduction in training time compared to its QA baseline counterpart. The effectiveness of our system stems from fine-tuning the BERT model twice, separately for span detection and classification. The source code can be found at https://github.com/c3sr/split-ner .

## introduction named entity recognition (ner) is a foundational task for a variety of applications like question answering and machine translation @xcite . traditionally, ner has been seen as a sequence labeling task where a model is trained to classify each token of a sequence to a predefined class @xcite @xcite @xcite @xcite . recently, there has been a new trend of formulating ner as span prediction problem @xcite @xcite @xcite , where a model is trained to jointly perform span boundary detection and multiclass classification over the spans. another trend is to formulate ner as a question answering (qa) task @xcite , where the model is given a sentence and a query corresponding to each entity type. the model is trained to understand the query and extracts mentions of the entity type as answers. while these new frameworks have shown improved results, both approaches suffer from a high computational cost: span-based ner systems consider all possible spans (i.e., n 2 (quadratic) spans for a sentence with n tokens) and the qa-based system multiplies each input sequence by the number of entity types resulting in n×t input sequences for n sentences and t entity types. in this work, we borrow the effectiveness of span-based and qa-based techniques and make it more efficient by breaking (splitting up) the ner task into a two-step pipeline of classification tasks. in essence, our overall approach comes under the span-based ner paradigm, and each sub-task is formulated as a qa task inspired by the higher accuracy offered by the qa framework. the first step, span detection performs token-level classification to extract mention spans from text, irrespective of entity type and the second step, span classification classifies the extracted spans into their corresponding entity type, thus completing the ner task. unlike other span-based ner techniques which are quadratic in terms of sequence length, our span detection process is linear. compared to other qa-based techniques which query for all entity types in each sentence, our span classification queries each sentence only once for each entity mention in the sentence. this makes it highly efficient for datasets with large number of entity types like ontonotes5.0. ## experimental results we demonstrate the effectiveness of our method in terms of performance and latency. ## related work in recent years, deep learning has been increasingly applied for ner @xcite , a popular architecture being cnn-lstm-crf @xcite and bert @xcite . @xcite propose a qa-based setup for ner using one model for both span detection and classification. li et al. (2020b); jiang et al. (2020); ouchi et al. (2020); fu et al. (2021); zhu and li (2022) perform ner as a span prediction task. however, they enumerate all possible spans in a sentence leading to quadratic complexity w.r.t. sentence length. our model does a token-level classification and hence is linear. xu et al. (2021) propose a syn-lstm setup leveraging dependency tree structure with pre-trained bert embeddings for ner. yan et al. (2021) propose a generative framework leveraging bart (lewis et al., 2020) for ner. yu et al. (2020) propose a biaffine model utilizing pretrained bert and fasttext (bojanowski et al., 2017) embeddings along with character-level cnn setup over a bi-lstm architecture. all of these models report good performance on ontonotes5.0, however, using bert-large architecture. nguyen and vu (2020) propose the bertweet model by training bert on a corpus of english tweets and report good performance on wnut17. wang et al. (2021) leverage external knowledge and a cooperative learning setup. on bionlp13cg, crichton et al. (2017) report 78.90 f1 in a multi-task learning setup and neumann et al. (2019) report 77.60 using the scispacy system. splitner(qa-qa) outperforms both of these by a large margin. ## conclusion using the qa-framework for both span detection and span classification, we show that this division of labor is not only effective but also significantly efficient through experiments on multiple crossdomain datasets. through this work, we open up the possibility of breaking down other complex nlp tasks into smaller sub-tasks and fine-tuning large pretrained language models for each task.