Focusing, Bridging and Prompting for Few-shot Nested Named Entity Recognition (2023)

Abstract: Few-shot named entity recognition (NER), identifying named entities with a small number of labeled data, has attracted much attention. Frequently, entities are nested within each other. However, most of the existing work on few-shot NER addresses flat entities instead of nested entities. To tackle nested NER in a few-shot setting, it is crucial to utilize the limited labeled data to mine unique features of nested entities, such as the relationship between inner and outer entities and contextual position information. Therefore, in this work, we propose a novel method based on focusing, bridging and prompting for few-shot nested NER without using source domain data. Both focusing and bridging components provide accurate candidate spans for the prompting component. The prompting component leverages the unique features of nested entities to classify spans based on soft prompts and contrastive learning. Experimental results show that the proposed approach achieves state-of-the-art performance consistently on the four benchmark datasets (ACE2004, ACE2005, GENIA and KBP2017) and outperforms several competing baseline models on F1-score by 9.33% on ACE2004, 6.17% on ACE2005, 9.40% on GENIA and 5.12% on KBP2017 on the 5-shot setting.

## introduction named entity recognition (ner), aiming at identifying the spans of text and classifying them into pre-defined entity categories, is a fundamental task in natural language processing @xcite . ner serves as a crucial component for many downstream tasks such as information extraction, sentiment analysis and other nlp applications @xcite . few-shot ner, focusing on named entity recognition with a small number of labeled data, has attracted much attention. frequently, entities are * corresponding author. nested within each other as shown in figure 1 (a). however, most of the existing work on few-shot ner addresses flat entities instead of nested entities. approaches for few-shot flat ner can mainly be divided into three categories: sequence-labelingbased, generative-based and span-based methods. sequence-labeling-based mehtods treat ner as sequence labeling that assigns a tag for each token using the bio or io tagging scheme @xcite @xcite . generativebased methods autoregressively generate the entity types or the pointer index sequence directly @xcite @xcite . span-based methods enumerate text spans in the input text and classify each span based on its corresponding template score @xcite , or the similarity between the span representation and the anchor @xcite @xcite . directly applying current few-shot flat ner methods to nested named entities suffers from some weaknesses. for sequence-labeling-based methods, extra strategies such as layering and concatenating the nested entity's multiple labels into one label @xcite are needed. such adaptation lacks flexibility and makes the already scarce supervision signal even more sparse. generative-based methods can directly handle nested entities. however, due to the auto-regressive generation manner, the optimiza-tion objective is not consistent with the ner task, resulting in some biases learned by the model during the training process @xcite . in addition, such biases are more difficult to eliminate with limited labeled data. by enumerating all the text spans, span-based nested ner can be converted into flat ner, which seems promising. however, such adaptation faces two challenges. first, it is crucial to utilize the relationship between inner and outer entities in nested ner, which is usually ignored in the previous work. some types of entities in medical-related datasets are prone to be nested. as shown in figure 1 (b) , in genia, the frequencies of the entities of protein type being nested with the entities of dna type are nearly five times higher than that with the entities of rna type. secondly, the same mention may have different types in polysemy scenarios. therefore, it is necessary to capture local features and precisely model contextual information. to address the issues mentioned above, we propose a novel span-based method based on focusing, bridging and prompting (fit) for few-shot nested ner without using source domain data. in the focusing stage, inspired by the io tagging scheme of sequence-labeling-based methods, each token is tagged whether a part of an entity or not. then entity-concentrated parts can be obtained by concatenating continuous tokens marked with the i-tag. in the bridging stage, for each entityconcentrated part, all spans obtained by enumerating are chosen as candidate spans and filtered according to the boundary score of each candidate span. the bridging stage acts as a bridge connecting the flat entity-concentrated parts with nested ner since nested entities can be obtained by enumerating. in the prompting stage, to make use of the relationship information between nested entities and contextual position information, adversarial prompt-based span classification is proposed. the soft prompts directly before and after the span are inserted to make full use of the contextual position information near the span for classification. moreover, contrastive learning is employed to shorten the distance between sentence representations to reduce the interference caused by soft prompts. in this way, we preserve the potential connections between nested entities. our main contributions are as follows: ## method in this section, we will first introduce the task definition of nested ner, then describe the details of fit. finally, the training objective is introduced. ## experiments in this section, we conduct experiments on four nested ner datasets to evaluate the effectiveness of the proposed method. ## time complexity theoretically, the number of possible spans in a sentence of length n is n (n +1) 2 . if we classify almost all spans into corresponding categories, it will lead to a high computational cost with o(n 2 ) time complexity. however, the focusing stage makes the model only focus on the entity-concentrated part, reducing the time complexity. although in the worst case, the model keeps the whole sentence as an entity-concentrated part, generating n (n +1) 2 candidate nested spans. the number of candidate spans is reduced as some partial overlap spans are discarded according to the boundary scores. we also evaluate the efficiency of fit. in the 5-shot setting of the ace2004 dataset, compared with few-shot span-based methods see-few training that takes 159.39s, the fit takes 122.37s for the same 35 epochs, which leads to approximately 23.23% speedup. in the inference phase, fit also spends 31.99ms for each sample on average, which is 15.50% faster than other results-competitive methods. time usage on four datasets can be found in appendix c. ## discussion the f 1 scores of 10 sets of 20-shot data sampled on the ace2005 dataset are compared in figure 3 . the horizontal coordinate is sorted in ascending order by the nested ratio (the lower bound is 22.86%, and the upper bound is 42.14%). the nested ratio of each set can be found in appendix a.1. it shows that under a single data set, the performance of the model is more closely related to the quality of the sampled data rather than the nested ratio. nevertheless, fit works better than other methods. to further explore the effect of the different nested ratios of the training sets on fit, we randomly sample 200 sets of 20-shot data from the ace2005 dataset and preserve sets that satisfy specific nested ratios. finally, 50 sets are kept and divided into 5 groups. (note that each group contains 10 sets and 5 groups corresponding to nested ratios of 17-24%, 20-34%, 40-44%, 60-64%, and 70-74%.) as shown in figure 4 , the f 1 score tends to decrease as the nested ratio increases. however, the e nested maintains a decreasing trend while the e f lat increases. it can be seen that the increase in the nested ratio may help the model learn nested entities better, and most of the decrease in f 1 is due to the misjudgment of flat entities. ## conclusion in this work, we propose a span-based method for few-shot nested ner without using source domain data. first, the candidate nested spans are generated by the focusing and bridging components. then the adversarial prompt-based span classification method is proposed to classify candidate spans into the corresponding categories. our proposed method, fit, can make full use of the unique features of nested entities while reducing the computational cost and the impact of low-quality candidate spans. experimental results show that our method achieves state-of-the-art performance consistently on the four benchmark datasets (ace2004, ace2005, genia, and kbp2017), and outperforms several competing baseline models on f1score and the error rates of nested entities.