SlowGuess/NAACL_2022_OCR huggingface dataset:
Table 1: F1 scores with standard deviations on FewNERD. The best results are in boldface.
Models	We	Mu	Pl	Bo	Se	Re	Cr	Avg.
1-SHOT	TransferBERT	55.82±2.75	38.01±1.74	45.65±2.02	31.63±5.32	21.96±3.98	41.79±3.81	38.53±7.42	39.06±3.86
MN+BERT	21.74±4.60	10.68±1.07	39.71±1.81	58.15±0.68	24.21±1.20	32.88±0.64	69.66±1.68	36.72±1.67
ProtoBERT	46.72±1.03	40.07±0.48	50.78±2.09	68.73±1.87	60.81±1.70	55.58±3.56	67.67±1.16	55.77±1.70
Ma2021	-	-	-	-	-	-	-	69.3(unk)
L-TapNet+CDT	71.53±4.04	60.56±0.77	66.27±2.71	84.54±1.08	76.27±1.72	70.79±1.60	62.89±1.88	70.41±1.97
ESD (Ours)	78.25±1.50	54.74±1.02	71.15±1.55	71.45±1.38	67.85±0.75	71.52±0.98	78.14±1.46	70.44±0.47
5-SHOT	TransferBERT	59.41±0.30	42.00±2.83	46.07±4.32	20.74±3.36	28.20±0.29	67.75±1.28	58.61±3.67	46.11±2.29
MN+BERT	36.67±3.64	33.67±6.12	52.60±2.84	69.09±2.36	38.42±4.06	33.28±2.99	72.10±1.48	47.98±3.36
ProtoBERT	67.82±4.11	55.99±2.24	46.02±3.19	72.17±1.75	73.59±1.60	60.18±6.96	66.89±2.88	63.24±3.25
Retriever	82.95(unk)	61.74(unk)	71.75(unk)	81.65(unk)	73.10(unk)	79.54(unk)	51.35(unk)	71.72(unk)
ConVEx	71.5(unk)	77.6(unk)	79.0(unk)	84.5(unk)	84.0(unk)	73.8(unk)	67.4(unk)	76.8(unk)
Ma2021	89.39(unk)	75.11(unk)	77.18(unk)	84.16(unk)	73.53(unk)	82.29(unk)	72.51(unk)	79.17(unk)
L-TapNet+CDT	71.64±3.62	67.16±2.97	75.88±1.51	84.38±2.81	82.58±2.12	70.05±1.61	73.41±2.61	75.01±2.46
ESD (Ours)	84.50±1.06	66.61±2.00	79.69±1.35	82.57±1.37	82.22±0.81	80.44±0.80	81.13±1.84	79.59±0.39
Table 2: F1 scores with standard deviations on 7 domains of SNIPS. The best results are in **boldface**. 'unk' denotes methods that do not report deviations in their paper. Baselines of 1-shot and 5-shot settings are different since ConVEx and Retriever do not report the 1-shot results in their paper.
as the validation domain, and train the model on source domains  $\mathcal{D} - \{\mathcal{D}_i,\mathcal{D}_j\}$ . In the sampling task of SNIPS, all classes have  $K$  annotated examples in the support set, but the number of them  $(N)$  is not fixed. The few-shot slot filling task in each domain of SNIPS has two settings, 1-shot and 5-shot. FSSL models are trained and evaluated on tasks sampled from the source and target domain, respectively. To ensure the fairness, we use the public sampled data provided by (Ding et al., 2021) for FewNERD² and data provided by (Hou et al., 2020) for SNIPS, to train and evaluate our model.
Parameter Settings Following previous methods (Hou et al., 2020; Ding et al., 2021), we use uncased BERT-base as our encoder. We use Adam (Kingma and Ba, 2015) as our optimizer. We set the dropout ratio (Srivastava et al., 2014) to 0.1. The maximum span length  $L$  is set to 8. For BSNMS, the beam size  $b$  is 5. Because FewNERD and
SNIPS do not have nested instances, we set the threshold to filter false positive spans  $\delta$  to 0.1, the threshold to decay span scores  $k$  to 1e-5 and the decay ratio  $u$  to 1e-5 to force the refining results have no nested spans. More details of our parameter settings are provided in Appendix A.
Evaluation Metrics For FewNERD, following (Ding et al., 2021), we report the micro F1 over all test tasks, and the average result of 5 different runs. For SNIPS, following (Hou et al., 2020), we first calculate micro F1 score for each test episode (an episode contains a number of test tasks), and then report the average F1 score for all test episodes as the final result. We report the average result of 10 different runs the same as (Hou et al., 2020).
Baselines For systematic comparisons, we introduce a variety of baselines, including ProtoBERT (Ding et al., 2021; Hou et al., 2020), NNShot (Ding et al., 2021), StructShot (Ding et al., 2021), TransferBERT (Hou et al., 2020), MN+BERT (Hou et al., 2020), L-TapNet+CDT (Hou et al., 2020), Retriever (Yu et al., 2021), ConVEx (Henderson
2The FewNERD data we used is from https://cloud.tsinghua.edu.cn/f/0e38bd108d7b49808cc4/?dl=1, which corresponds to the results reported in https://arxiv.org/pdf/2105.07464v6.pdf.
Ablation Models	F1
ESD	61.7±1.3
r.m. intra span attention (ISA)	61.0±0.4
r.m. cross span attention (CSA)	59.1±1.1
r.m. instance span attention (INSA)	58.2±1.3
r.m. O partition (OP)	60.6±1.3
r.m. beam soft-nms (BSNMS)	57.3±1.4
Table 3: The effect of our proposed mechanisms on the validation set of SNIPS (1-shot, domain "We"). We report the average result of 3 different runs with standard deviations. r.m. denotes remove.
and Vulić, 2021) and Ma2021 (Ma et al., 2021). Please refer to the Appendix A for more details.
Table 1 illustrates the results on FewNERD. As is shown, among all task settings, ESD consistently outperforms ProtoBERT, NNShot and StructShot by a large margin. For example, compared with StructShot, ESD achieves 11.17 and 10.60 average F1 improvement on INTRA and INTER, respectively. Table 2 shows the results on SNIPS. In the 1-shot setting, L-TapNet+CDT is the previous best method. Compared with L-TapNet+CDT, ESD achieves comparable results and 4.58 F1-scores improvement in the 1-shot and 5-shot settings, respectively. We think the reason is that the information benefits brought by our cross span attention mechanism in 1-shot setting is much less than that in 5-shot setting. In addition, compared with L-TapNet+CDT, ESD performs more stable, and also has a better model efficiency (Please refer to Section 6.3). In 5-shot setting, Ma2021 is previous best method, and ESD outperforms it 1.14 and 0.42 F1-scores in 1-shot and 5-shot settings, respectively. These results prove the effectiveness of ESD in few-shot sequence labeling.
To illustrate the effect of our proposed mechanisms, we conduct ablation studies by removing one component of ESD at a time. Table 3 shows the results on the validation set of SNIPS (1-shot, domain "We"). Firstly, we remove ISA or CSA, which means the span cannot be aware of other spans within the same sentence or spans from other sentences. As shown in Table 3, the average F1 scores drop 0.7 and 2.6 without ISA and CSA, respec
Figure 4: The performance drops in nested situations when increasing  $r_{\text{nest}}$ .  $\Delta \mathrm{F}1\%$  is the F1 reduction percent over  $r_{\text{nested}} = 0$ .
tively. These results suggest that our span enhancing module with ISA and CSA is effective. In addition, CSA is more effective than ISA, since CSA can enhance query spans with whole support set spans, while ISA enhances spans only with other spans in the same setence. CSA brings much more information than ISA. Secondly, when we remove INSA and achieve the prototypical representation of a class through an average operation, the average F1 score would drop 3.5. When we do not consider the sub-classes of O-type spans (r.m. OP), the average F1 score would drop 1.1. These results show that our span prototypical module is necessary. At last, the result without BSNMS suggests the importance of our post-processing algorithm in this span-level few-shot labeling framework. Moreover, with BSNMS, ESD can also easily adapt to the nested situation.
6.2 Robustness in the Nested Situation
Sequence labeling tasks such as NER can have nested situations. For example, in the sentence "Isaac Newton studied at Cambridge University", where "Cambridge" is a location and "Cambridge University" is an organization. However, both FewNERD and SNIPS do not annotate nested examples. To explore the robustness of ESD on such a challenging but common situation, we construct FewNERD-nested, which has the same training tasks as FewNERD-INTRA, but different test tasks with a nested ratio  $r_{\text{nested}}$ . In FewNERD-nested, we sample each test task either from FewNERD or from GENIA (Ohta et al., 2002) with the probability  $1 - r_{\text{nested}}$  and  $r_{\text{nested}}$ , respectively, and all tasks sampled from GENIA are guaranteed to have the query with nested entities. We sample validation tasks with nested instances from
Models	F1
ESD (with BSNMS)	31.25
ESD (with SoftNMS)	31.09
ESD (with Beam Search)	28.94
ESD (without Post-processing)	30.34
Table 4: Comparison between different post-processing methods in the nested situation with  $r_{\text{nested}} = 1$ .
Models	#Para.	Inference Time
1-SHOT	5-SHOT
ESD	112M	8.53 ms	18.47 ms
ProtoBERT	110M	3.13 ms	5.27 ms
L-TapNet+CDT	110M	24.67 ms	54.19 ms
Table 5: The parameter number and inference time per task of ESD and L-TapNet+CDT in the domain "We" of SNIPS. Although ESD is slower than ProtoBERT, ESD outperforms ProtoBERT by 31.53 and 16.68 F1 scores in 1- and 5-shot settings with a bearable latency.
ACE05 (Walker et al., 2006) to tune  $k$ ,  $u$  and  $\delta$  in BSNMS. Please refer to the Appendix B for more details about FewNERD-nested. Figure 4 shows the results of ESD and several typical baselines in FewNERD-nested with different  $r_{nested}$ . When  $r_{nested}$  increases, ESD is more stable than previous methods, since ESD with BSNMS can easily extend to nested tagging cases without any extra training while previous methods are incapable of. In addition, we also compare different post-processing methods when  $r_{nested} = 1$ . As shown in Table 4, the beam search method can not handle the nested situation, and thus it harms the performance. SoftNMS (Shen et al., 2021) and BSNMS can both improve the model performance. However, when incorporating beam search into SoftNMS, the model can be more flexible and avoid some local optimal post-processing results achieved by SoftNMS (e.g., spans 1,3 and 4 in Figure 3), and thus BSNMS outperforms SoftNMS<sup>3</sup>.
6.3 Model Efficiency
Compared with the token-level models, ESD needs to enumerate all spans within the length  $L$  for a sentence. Therefore, the number of spans is approximately  $L$  times that of tokens, which may bring extra computation overhead. To evaluate the efficiency of ESD, we compare the average inference
Models	F1	Total	FP-Span	FP-Type
ESD	59.29	9.4k	72.8%	27.2%
ProtoBERT	38.83	30.4k	86.7%	13.3%
NNShot	47.24	21.7k	84.7%	15.3%
StructShot	51.88	14.5k	80.0%	20.0%
Table 6: Error analysis of 5 way  $1\sim 2$  shot on FewNERD-INTER. 'FP-Span' denotes extracted entities with the wrong span boundary, and 'FP-Type' represents extracted entities with the right span boundary but the wrong entity type. 'Total' denotes the total wrong prediction of two types.
time per task of ESD (including the BSNMS post-processing process), L-TapNet+CDT (the state-of-the-art token-level baseline model with the open codebase in the SNPIS) and ProtoBERT (an extremely simple token-level baseline). As shown in Table 5, with exactly the same hardware setting, in the domain "We" of SNIPS 1-shot setting, ESD (avg.  $8.53\mathrm{ms}$  per task) is nearly 3 times faster than L-TapNet+CDT (avg.  $24.67\mathrm{ms}$  per task). We see a similar tendency in the 5-shot setting. Although ESD (avg.  $8.53$  and  $18.47\mathrm{ms}$  in 1- and 5-shot per task) is slower than ProtoBERT (avg.  $3.13$  and  $5.27\mathrm{ms}$ ), it outperforms ProtoBERT by 31.53 and 16.68 F1 scores in 1- and 5-shot settings (reported in Table 2) with a bearable latency. Besides the inference time, we also compare the parameter number of these models. As is shown, the added parameter scale (span enhancing and prototypical modules) is very small (2M) compared with the ProtoBERT and L-TapNet+CDT (110M). These results show that ESD has an acceptable efficiency.
6.4 Error Analysis
To further explore what types of errors the model makes in detail, we divide error of model prediction into 2 categories, 'FP-Span' and 'FP-Type'. As shown in Table 6, ESD outperforms baselines and has much less false positive prediction errors. 'FP-Span' is the major prediction error for all models, showing that it is hard to locate the right span boundary in FSSL for existing models. Therefore, we should pay more attention to the span recognition in the future work. However, compared with previous methods, ESD has less ratio of the 'FP-span' error. We think the reason is that our span-level matching framework with a series of span-related procedures has a better perception of the entity and slot spans than that of our baselines.
ESD is also more robust than baselines in the noisy situation. Please refer to Appendix C for details.
In this paper, we propose ESD, an enhanced span-based decomposition model for few-shot sequence labeling (FSSL). To overcome the drawbacks of previous token-level methods, ESD formulates FSSL as a span-level matching problem, and decomposes it into a series of span-related procedures, mainly including span representation, class prototype aggregation and span conflicts resolution for a better span matching. Extensive experiments show that ESD achieves the state-of-the-art performance on two popular few-shot sequence labeling benchmarks and that ESD is more robust than previous models in the noisy and nested situation.
The authors would like to thank the anonymous reviewers for their thoughtful and constructive comments. This paper is supported by the National Key Research and Development Program of China under Grant No. 2020AAA0106700, the National Science Foundation of China under Grant No.61936012 and 61876004, and NSFC project U19A2065.
Athiwaratkun, B.; Nogueira dos Santos, C.; Krone, J.; and Xiang, B. 2020. Augmented Natural Language for Generative Sequence Labeling. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 375-385. Online: Association for Computational Linguistics.
Ba, J. L.; Kiros, J. R.; and Hinton, G. E. 2016. Layer normalization. arXiv preprint arXiv:1607.06450.
Bodla, N.; Singh, B.; Chellappa, R.; and Davis, L. S. 2017. Soft-NMS-improving object detection with one line of code. In Proceedings of the IEEE international conference on computer vision, 5561-5569.
Coucke, A.; Saade, A.; Ball, A.; Bluche, T.; Caulier, A.; Leroy, D.; Doumouro, C.; Gisselbrecht, T.; Caltagirone, F.; Lavril, T.; et al. 2018. Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces. arXiv preprint arXiv:1805.10190.
Cui, L.; Wu, Y.; Liu, J.; Yang, S.; and Zhang, Y. 2021. Template-Based Named Entity Recognition Using BART. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, 1835-1845. Online: Association for Computational Linguistics.
Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019. BERT: Pre-training of Deep Bidirectional
Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171-4186. Minneapolis, Minnesota: Association for Computational Linguistics.
Ding, N.; Xu, G.; Chen, Y.; Wang, X.; Han, X.; Xie, P.; Zheng, H.; and Liu, Z. 2021. Few-NERD: A Few-shot Named Entity Recognition Dataset. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 3198-3213. Online: Association for Computational Linguistics.
Fritzler, A.; Logacheva, V.; and Kretov, M. 2019a. Few-shot classification in named entity recognition task. In Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing, 993-1000.
Fritzler, A.; Logacheva, V.; and Kretov, M. 2019b. Few-shot classification in named entity recognition task. In Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing, 993-1000.
Geng, R.; Li, B.; Li, Y.; Zhu, X.; Jian, P.; and Sun, J. 2019. Induction Networks for Few-Shot Text Classification. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 3904-3913. Hong Kong, China: Association for Computational Linguistics.
Han, X.; Zhu, H.; Yu, P.; Wang, Z.; Yao, Y.; Liu, Z.; and Sun, M. 2018. FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 4803-4809. Brussels, Belgium: Association for Computational Linguistics.
He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770-778.
Henderson, M.; and Vulic, I. 2021. ConVEx: Data-Efficient and Few-Shot Slot Labeling. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 3375-3389. Online: Association for Computational Linguistics.
Hochreiter, S.; Younger, A. S.; and Conwell, P. R. 2001a. Learning to learn using gradient descent. In International Conference on Artificial Neural Networks, 87-94. Springer.
Hochreiter, S.; Younger, A. S.; and Conwell, P. R. 2001b. Learning to learn using gradient descent. In International Conference on Artificial Neural Networks, 87-94. Springer.
Hou, Y.; Che, W.; Lai, Y.; Zhou, Z.; Liu, Y.; Liu, H.; and Liu, T. 2020. Few-shot Slot Tagging with Collapsed Dependency Transfer and Label-enhanced Task-adaptive Projection Network. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 1381-1393. Online: Association for Computational Linguistics.
Kingma, D. P.; and Ba, J. 2015. Adam: A Method for Stochastic Optimization. CoRR, abs/1412.6980.
Kulis, B.; et al. 2013. Metric Learning: A Survey. Foundations and Trends® in Machine Learning, 5(4): 287-364.
Lewis, M.; Liu, Y.; Goyal, N.; Ghazvininejad, M.; Mohamed, A.; Levy, O.; Stoyanov, V.; and Zettlemoyer, L. 2020. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 7871-7880.
Ma, J.; Yan, Z.; Li, C.; and Zhang, Y. 2021. Frustratingly Simple Few-Shot Slot Tagging. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, 1028-1033. Online: Association for Computational Linguistics.
Ohta, T.; Tateisi, Y.; Kim, J.-D.; Mima, H.; and Tsujii, J. 2002. The GENIA corpus: An annotated research abstract corpus in molecular biology domain. In Proceedings of the human language technology conference, 73-77. Citeseer.
Shen, Y.; Ma, X.; Tan, Z.; Zhang, S.; Wang, W.; and Lu, W. 2021. Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2782-2794.
Snell, J.; Swersky, K.; and Zemel, R. 2017. Prototypical networks for few-shot learning. In Proceedings of the 31st International Conference on Neural Information Processing Systems, 4080-4090.
Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; and Salakhutdinov, R. 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1): 1929-1958.
Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L.; and Polosukhin, I. 2017. Attention is all you need. In Advances in neural information processing systems, 5998-6008.
Vinyals, O.; Blundell, C.; Lillicrap, T.; kavukcuoglu, k.; and Wierstra, D. 2016. Matching Networks for One Shot Learning. In Lee, D.; Sugiyama, M.; Luxburg, U.; Guyon, I.; and Garnett, R., eds., Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc.
Walker, C.; Strassel, S.; Medero, J.; and Maeda, K. 2006. ACE 2005 multilingual training corpus. Linguistic Data Consortium, Philadelphia, 57: 45.
Wang, P.; Xun, R.; Liu, T.; Dai, D.; Chang, B.; and Sui, Z. 2021a. Behind the Scenes: An Exploration of Trigger Biases Problem in Few-Shot Event Classification. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 1969-1978.
Wang, Y.; Mukherjee, S.; Chu, H.; Tu, Y.; Wu, M.; Gao, J.; and Awadallah, A. H. 2021b. Meta Self-training for Few-shot Neural Sequence Labeling. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 1737-1747.
Wang, Y.; Yao, Q.; Kwok, J. T.; and Ni, L. M. 2020. Generalizing from a few examples: A survey on few-shot learning. ACM Computing Surveys (CSUR), 53(3): 1-34.
Yang, Y.; and Katiyar, A. 2020. Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 6365-6375. Online: Association for Computational Linguistics.
Yoon, S. W.; Seo, J.; and Moon, J. 2019. Tapnet: Neural network augmented with task-adaptive projection for few-shot learning. In International Conference on Machine Learning, 7115-7123. PMLR.
Yu, D.; He, L.; Zhang, Y.; Du, X.; Pasupat, P.; and Li, Q. 2021. Few-shot Intent Classification and Slot Filling with Retrieved Examples. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 734-749. Online: Association for Computational Linguistics.
We compare ESD with a variety of baselines as follows:
- TransferBERT (Hou et al., 2020) is a fine-tuning based model, which is a direct application of BERT (Devlin et al., 2019) to the few-shot sequence labeling.
- ConVEx (Henderson and Vulic, 2021) is a fine-tuning based model, which is first pretrained on the Reddit corpus with the sequence labeling objective tasks and then fine-tuned on the source domain and target domain annotated data for final few shot sequence labelingm
- Ma2021 (Ma et al., 2021) formulates sequence labeling as the machine reading comprehension problem, and proposes some questions to extract slots in the query sentence.
- ProtoBERT (Fritzler, Logacheva, and Kretov, 2019b) predicts the query labels according to the similarity of BERT hidden states of support set and query tokens.
- Matching Net (MN) + BERT (Hou et al., 2020) is similar to ProtoBERT. The only difference is that MN uses the matching network (Vinyals et al., 2016) for token classification.
- L-TapNet-CDT (Hou et al., 2020) utilizes the task-adaptive projection network (Yoon, Seo, and Moon, 2019), pair-wise embedding and collapsed dependency transfer mechanisms to do classification.
- NNShot (Yang and Katiyar, 2020) is similar to ProtoBERT, while it makes the prediction based on the nearest neighbor.
- StructShot (Yang and Katiyar, 2020) adopts an additional Viterbi decoder during the inference phase on top of NNShot.
- Retriever (Yu et al., 2021) is a retrieval based method which does classification according to the most similar example in the support set.
A.2 Parameter Setting
In our implementation, we utilize BERT-base-uncased as our backbone encoder the same as (Hou et al., 2020; Yu et al., 2021; Ding et al., 2021). We use Adam (Kingma and Ba, 2015) as our optimizer. In FewNERD, the learning rate is  $2e - 5$  for BERT encoder and  $5e - 4$  for other modules. In the 1-shot setting of SNIPS, for the domain "Mu", the learning rate is  $5e - 6$  for BERT encoder and  $1e - 4$  for other modules. For the domain "Bo", the learning rate is  $1e - 5$  for BERT encoder and  $1e - 4$  for other modules. For other settings of SNIPS, the learning rate is  $5e - 5$  for BERT encoder and  $5e - 4$  for other modules. We set the dropout ratio (Srivastava et al., 2014) to 0.1. The dimension of span representation  $d$  and the maximum span length  $L$  is set to 100 and 8, respectively. For BSNMS, the beam size  $b$  is 5. Since these is no nested instances in FewNERD and SNIPS, we set the threshold to filter false positive spans  $\delta$  to 0.1, the threshold to decay span scores  $k$  to  $1e - 5$  and the decay ratio  $u$  to  $1e - 5$  to force the refining results have no nested spans. We use the grid search to search our hyperparameters, and the scope of each hyperparameter are included in Table 7. We train our model on a single NVIDIA A40 GPU with 48GB memory.
learning rate	[5e-5, 1e-4, 3e-4, 5e-4]
dropout	[0.1, 0.3, 0.5]
bert learning rate	[5e-6, 1e-5, 2e-5, 3e-5, 5e-5]
span dimension	[50, 100, 150, 200]
beam size	[1, 3, 5, 7]
B FewNERD-nested