SlowGuess/NAACL_2022_OCR huggingface dataset:
In addition, we present the model performance on CBLUE-CMeEE in Table 2. Considering there are no available lexicons for this task, we only compare RICON with typical models. As shown in this table, RICON outperforms the strong BERT-Biaffine model with a 3.28 F1 score improvement over 9 types. It is remarkable progress in this challenging dataset. Meanwhile, we provided the result of nested symptom type. RICON performs much better than BERT-Biaffine with a 5.81 F1 improvement. This observation also denotes that our RICON also applies to nested NER.
We conduct abundant ablation studies on OntoNotes V4.0 and V5.0 from module and implementation perspectives in Table 3 and 4. Vanilla in tables is built from RICON by removing orthogonality space and regularity-agnostic module, and omitting to capture regularity features and integrate it in the regularity-aware module.
From the results in Table 3, we can observe that: (1) When applying regularity-agnostic module to the vanilla, the performances improve by
Module	OntoNotes V4.0	OntoNotes V5.0
P	R	F1	P	R	F1
Vanilla	81.08	84.17	82.59	77.87	82.04	79.42
+Reg-agnostic	81.18	84.77	82.80	78.32	81.48	79.90
+Reg-aware	82.49	83.86	83.16	79.28	80.90	80.07
+Reg-aware & agnostic	81.72	84.89	83.28	79.24	81.48	80.33
RICON (Ours)	81.95	84.78	83.33	79.26	81.64	80.43
0.21 and 0.48 respectively, showing the effectiveness of this module. (2) When the vanilla equips with regularity-aware module, the F1 scores significantly improve by 0.57 and 0.65 respectively, which verifies that regularity plays a significant role in entity recognition. (3) After combining regularity-aware and regularity-agnostic modules, we achieve further improvements, which indicates that two modules can mutually reinforce each other. (4) The orthogonality space is a valid method according to the further F1 score improvements.
Furthermore, we notice that adding the regularity-aware module significantly increases the Precision (1.41 on both datasets, Vanilla vs Vanilla+Reg-aware) but reduces the Recall (0.31 and 1.04 respectively), which conforms to that focusing on regularity feature would reinforce the type prediction, while missing several spans that are supposed to be entities. Nevertheless, this situation can be remedied by the regularity-agnostic module and the Recall improved 1.03 and 0.58, respectively (Vanilla+Reg-aware vs Vanilla+Reg-aware & agnostic). This result also meets our motivation that regularity-agnostic module can reinforce the entity boundary detection.
As shown in Table 4, there are several alternative ways to extract regularity information instead of linear attention used in this paper, such as mean-pooling, max-pooling, or more complex multi-head self-attention (Vaswani et al., 2017), but these methods all perform worse. It is one future direction to explore how to obtain regularity by a more sophisticated architecture. However, considering the model complexity and performance, we choose linear attention to capture regularity. In addition, replacing our devised gate mechanism with a simple concatenate or add operation both degrades the performance, denoting that gate mechanism is more efficient to integrate span feature and regularity feature. We also explored adding two MLPs separately to head and tail features when generating span features in the regularity-aware module. The experimental results prove that different fea
Table 3: Performance of modules on OntoNotes.
Implementation	Dataset (F1)
OntoNotes V4.0	OntoNotes V5.0
Vanilla+Reg-aware	83.16	80.07
Reg. feature by Mean-pooling	83.06 (-0.10)	79.97 (-0.10)
Reg. feature by Max-pooling	82.82 (-0.34)	79.79 (-0.28)
Reg. feature by Multi-Head	83.10 (-0.06)	79.86 (-0.21)
Gate replaced with Add	82.96 (-0.20)	79.94 (-0.13)
Gate replaced with Cat	82.80 (-0.36)	79.77 (-0.30)
Apply MLPs to head and tail	82.90 (-0.26)	79.67 (-0.40)
Vanilla	82.59 (-0.57)	79.42 (-0.65)
Table 4: Performance of variants on OntoNotes datasets.
ture space for span feature and regularity feature leads to worse performance.
In this section, We deeply analyze our proposed RICON from the following aspects.
4.6.1 Regularity: A Latent Adaptive Lexicon.
The lexicon-based methods focus on incorporating external word lexicons to improve the performance of character-based NER. The core concept of them is preserving all words which match a specific character and let the subsequent NER model determine which word to apply (Zhang and Yang, 2018; Ma et al., 2020). In our model, we calculate the regularity for each span, namely, all words containing a specific character are considered, and then the best word and corresponding regularity will be determined. In this sense, our explored regularity can be seen as a latent adaptive lexicon. Furthermore, this latent adaptive lexicon is more complete than external lexicons because all spans matching the specific character are considered, while lexicon-based methods only match a limited number of words. As shown in Table 1, the previous SOTA method BERT+Biaffine performs worse than lexicon-based methods, but our regularity-based method RICON outperforms the lexicon-based methods. Actually, our regularity-based method can further be combined with lexicon-based methods.
4.6.2 Performance vs. Entity Type.
We examine how regularity affects each entity type. As Figure 4 shows, 12 types of entities achieve better performance with the regularity. This result conforms to the fact that types like GPE, ORG, and DATE have strong regularity. Nevertheless, for the types with little regularity information, such as WORK_OF_ART and PERSON, immersed regularity leads to performance degradation. We notice that the MONEY type typically contains regularity but we do not observe an improvement in
Figure 4: The performance of 15 types of entities on OntoNotes V5.0. The types are sorted in descending order based on the proportion of entities of that type to the total. As the remaining 3 types on OntoNotes V5.0 only have less than 35 entities (0.05%) among all entities. To avoid the impact of labeling errors, we do not present them here.
#1 Sentence (Truncated)	据报道,从波罗的海三国撤回的俄罗斯军队......
Characters (Entity Included)	(Reportedly, Russian Army withdrawn from the three countries around Baltic Sea...)
波	罗	的	海
Gold Label	B-LOC	M-LOC	M-LOC	E-LOC
Vanilla	B-GPE	M-GPE	M-GPE	M-GPE
Vanilla + Reg-aware	B-LOC	M-LOC	M-LOC	E-LOC
Regularity weight	0.04	0.06	0.07	0.83
#2 Sentence (Truncated)	新闻分析:美国公司兼并为何愈演愈烈?
Characters (Entity Included)	(News analysis: why the mergers of American companies are intensifying?)
美	国	公	司
Gold Label	B-GPE	E-GPE	O	O
Vanilla + Reg-aware	B-ORG	M-ORG	M-ORG	M-ORG
Reg-aware + Reg-agnostic	B-GPE	E-GPE	O	O
Table 5: There examples from the Ontonotes V4.0 dataset. The label is organized in the form of BMES.
(a) F1 on OntoNotes V4.0
Figure 5: Performance vs. Entity Length
(b) F1 on OntoNotes V5.0
this category. This is, due to inconsistencies between the training and test dataset. For instance, the training data contains the abundant pattern "number+dollar", while only numbers exist in the test set. To remedy the excessive regularity, our RICON further utilizes a regularity-agnostic module to rectify the captured regularity. The above observations also inspire us to devise more elaborate NER for different entity types with various degree regularity properties in the future. Our regularity-aware module may also serve as a potential tool for evaluating the intensity of regularity.
4.6.3 Performance vs. Entity Length.
Figure 5 depicts the performance on the OntoNotes V4.0 and V5.0 datasets with different length of entities. From this figure, we can observe that our RICON consistently outperforms BERT-Biaffine
(Yu et al., 2020) when the entity length is longer than 2, which illustrates that the regularity information is helpful to predict the types for long entities. In contrast, BERT-Biaffine performs comparable to RICON when entity length is 2 as there are no additional character information except the head and tail representations.
4.6.4 Case study.
Table 7 shows two examples from OntoNotes V4.0. In the first example, the Vanilla misidentifies the entity type, while Vanilla+reg-aware learns regularity "XX+海" by the greatest weight 0.83 on "海", thus obtaining the accurate entity type. It is worth noting that regularity can capture more complex character compositions besides explicit patterns in the first example. More complex examples are presented in the appendix. In the second example, "美国公司" conforms to the regularity "XX + 公司" and is recognized as organization type by our Vanilla+Reg-aware model. After equipping with the regularity-agnostic module, we obtain the precise character boundary and relieve the excessive attention to regularity.
In this paper, we proposed a simple but effective method to explore the regularity information
for Chinese NER, dubbed as Regularity-Inspired reCOgnition Network (RICON). It contains a regularity-aware module to capture the internal regularity feature of each span, and a regularity-agnostic module to reinforce the entity boundary detection while avoid imposing excessive attention on regularity. The features of two modules are encouraged to be dissimilar by an orthogonality space restriction. Evaluation shows that RICON achieves the state-of-the-art performance on four datasets.
Chun Chen and Fang Kong. 2021. Enhancing entity boundary detection for better Chinese named entity recognition. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 20-25, Online. Association for Computational Linguistics.
Qiao Cheng, Juntao Liu, Xiaoye Qu, Jin Zhao, Jiaqing Liang, Zhefeng Wang, Baoxing Huai, Nicholas Jing Yuan, and Yanghua Xiao. 2021. Hacred: A largescale relation extraction dataset toward hard cases in practical applications. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2819-2831.
Kevin Clark and Christopher D. Manning. 2016. Improving coreference resolution by learning entity-level distributed representations. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 643-653, Berlin, Germany. Association for Computational Linguistics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Yingjie Gu, Xiaoye Qu, Zhefeng Wang, Baoxing Huai, Nicholas Jing Yuan, and Xiaolin Gui. 2021. Read, retrospect, select: An mrc framework to short text entity linking. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 12920-12928.
Tao Gui, Yicheng Zou, Qi Zhang, Minlong Peng, Jinlan Fu, Zhongyu Wei, and Xuanjing Huang. 2019. A lexicon-based graph neural network for Chinese NER. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages
1040-1050, Hong Kong, China. Association for Computational Linguistics.
Zan Hongying, Li Wenxin, Zhang Kunli, Ye Yajuan, Chang Baobao, and Sui Zhifang. 2020. Building a pediatric medical corpus: Word segmentation and named entity annotation. In Workshop on Chinese Lexical Semantics, pages 652-664.
Z. Huang, X. Wei, and Y. Kai. 2015. Bidirectional LSTM models for sequence tagging. Computer Science arXiv preprint arXiv:1508.01991.
S. Ji, S. Pan, E. Cambria, P. Marttinen, and P. S. Yu. 2020. A survey on knowledge graphs: Representation, acquisition and applications. Computer Science arXiv preprint arXiv:2002.00388.
Z. Jie and W. Lu. 2019. Dependency-guided lstmrcf for named entity recognition. arXiv preprint arXiv:1909.10148.
Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 260-270, San Diego, California. Association for Computational Linguistics.
Gina-Anne Levow. 2006. The third international Chinese language processing bakeoff: Word segmentation and named entity recognition. In Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing, pages 108-117, Sydney, Australia. Association for Computational Linguistics.
J. Li, A. Sun, and Y. Ma. 2020a. Neural named entity boundary detection. IEEE Transactions on Knowledge and Data Engineering, PP(99):1-1.
Xiaonan Li, Hang Yan, Xipeng Qiu, and Xuanjing Huang. 2020b. FLAT: Chinese NER using flat-lattice transformer. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6836-6842, Online. Association for Computational Linguistics.
Hongyu Lin, Yaojie Lu, Jialong Tang, Xianpei Han, Le Sun, Zhicheng Wei, and Nicholas Jing Yuan. 2020. A rigorous study on named entity recognition: Can fine-tuning pretrained model lead to the promised land? arXiv preprint arXiv:2004.12126.
W. Liu, X. Fu, Y. Zhang, and W. Xiao. 2021. Lexicon enhanced chinese sequence labelling using bert adapter. arXiv preprint arXiv:2105.07148.
Ruotian Ma, Minlong Peng, Qi Zhang, Zhongyu Wei, and Xuanjing Huang. 2020. Simplify the usage of lexicon in Chinese NER. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5951-5960, Online. Association for Computational Linguistics.
Xuezhe Ma and Eduard Hovy. 2016. End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1064-1074, Berlin, Germany. Association for Computational Linguistics.
Xue Mengge, Bowen Yu, Tingwen Liu, Yue Zhang, Erli Meng, and Bin Wang. 2020. Porous lattice transformer encoder for Chinese NER. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3831-3841, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227-2237, New Orleans, Louisiana. Association for Computational Linguistics.
Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Hwee Tou Ng, Anders Björkelund, Olga Uryupina, Yuchen Zhang, and Zhi Zhong. 2013. Towards robust linguistic analysis using OntoNotes. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning, pages 143-152, Sofia, Bulgaria. Association for Computational Linguistics.
Yongliang Shen, Xinyin Ma, Zeqi Tan, Shuai Zhang, Wen Wang, and Weiming Lu. 2021. Locate and label: A two-stage identifier for nested named entity recognition. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics.
Mohammad Golam Sohrab and Makoto Miwa. 2018. Deep exhaustive model for nested named entity recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2843-2849, Brussels, Belgium. Association for Computational Linguistics.
Dianbo Sui, Yubo Chen, Kang Liu, Jun Zhao, and Shengping Liu. 2019. Leverage lexical knowledge for Chinese named entity recognition via collaborative graph network. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3830-3840, Hong Kong, China. Association for Computational Linguistics.
Z. Tang, B. Wan, and L. Yang. 2020. Word-character graph convolution network for chinese named entity recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing, PP(99):1-1.
R. Weischedel, M. Palmer, M. Marcus, E. Hovy, S. Pradhan, L. Ramshaw, N. Xue, A. Taylor, J. Kaufman, and M. Franchini. 2011. Ontonotes release 4.0. LDC2011T03, Philadelphia, Penn.: Linguistic Data Consortium.
Congying Xia, Chenwei Zhang, Tao Yang, Yaliang Li, Nan Du, Xian Wu, Wei Fan, Fenglong Ma, and Philip Yu. 2019. Multi-grained named entity recognition. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1430-1440, Florence, Italy. Association for Computational Linguistics.
Juntao Yu, Bernd Bohnet, and Massimo Poesio. 2020. Named entity recognition as dependency parsing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6470-6476, Online. Association for Computational Linguistics.
Yue Zhang and Jie Yang. 2018. Chinese NER using lattice LSTM. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1554-1564, Melbourne, Australia. Association for Computational Linguistics.
Table 6 shows the detailed statistics of each dataset.
Datasets	Type	Train	Dev	Test
OntoNotes V4.0	Sentence	15.7K	4.3K	4.3K
Char	491.9K	200.5K	208.1K
Entity	12.8K	6.5K	7.2K
OntoNotes V5.0	Sentence	36K	6.1K	4.5K
Char	1197.5K	173.3K	147.4K
Entity	58.1K	8.5K	7.0K
MSRA	Sentence	46.4K	-	4.4K
Char	2169.9K	-	172.6K
Entity	69.7K	-	5.2K
CBLUE-CMeEE	Sentence	15.3K	5.0k	-
Char	825.0K	270.4K	-
Entity	62.0K	20.3K	-
Table 6: Statistics of datasets.
B More Case Study
B.1 Complex Regularity
Besides explicit patterns like the first example in Table 5, Table 7 shows a more complex form of regularity that our model can capture. In this example, the Vanilla+Reg-aware model pays highest attention weight 0.92 to important character "和" (and), and recognize that A and B are independent entities according to the regularity "A 和 B (A and B)". For comparison, the vanilla fails to
Sentence (Truncated)	铢德和年兴纺织，是台湾唯二上榜的公司。(RITEK and Nien Hsing Textiles are the only two companies on the list in Taiwan.)
Characters (Entity Included)	铢	德	和	年	兴	纺	织
Gold Label	B-ORG	E-ORG	O	B-ORG	M-ORG	M-ORG	E-ORG
Vanilla	O	B-ORG	M-ORG	M-ORG	M-ORG	M-ORG	E-ORG
Vanilla + Reg-aware	B-ORG	E-ORG	O	B-ORG	M-ORG	M-ORG	E-ORG
Regularity weight	7.5e-2	2.5e-6	9.2e-1	2.1e-6	8.9e-6	9.5e-4	3.1e-5
Table 7: An example on the Ontonotes V4.0 dataset. The label is organized in the form of BMES.
Sentence (Truncated)	Golden Entity / Type	Biaffine Prediction	RICON Prediction	Regularity
(1) 肺多叶病变显示...	肺多叶病变，Symptom	未识别	肺多叶病变，Symptom	XX+病变
Multi-lobed lung lesions showed that...	Multi-lobed lung lesions	N/A	Multi-lobed lung lesions
(2) 大片状融合性病变为主...	大片状融合性病变，Symptom	未识别	大片状融合性病变，Symptom	XX+lesion
Massive fusion lesions were the main...	Massive fusion lesion	N/A	Massive fusion lesion
(3) 增加肝脏病变...多数属...	肝脏病变，Symptom	未识别	肝脏病变，Symptom
Increase liver lesions, most of which...	liver lesions	N/A	liver lesions
(4) ...患儿可并发肝损害。	肝损害，Symptom	肝损害，Disease	肝损害，Symptom	XX+损害
...can be complicated with liver damage.	liver damage	liver damage	liver damage
(5) SARS患儿有部分出现心脏损害...	心脏损害，Symptom	心脏损害，Disease	心脏损害，Symptom	XX+damage
SARS children suffer from heart damage...	heart damage	heart damage	heart damage
(6) ...合并多脏器损害	多脏器损害，Symptom	多脏器损害，Disease	多脏器损害，Symptom
...complicated with multi-organ damage	multi-organ damage	multi-organ damage	multi-organ damage
Table 8: Cases study on the domain CBLUE-CMeEE dataset.
distinguish these two entities. This example further reveals that our regularity-aware module can discover more complex character compositions.
B.2 Case Study in Medical Domain
To further demonstrate the effectiveness of our RICON in Chinese NER, we present six examples of the CBLUE-CMeEE dataset from the medical domain. As shown in the first three examples in Table 8, the biaffine model fails to identify the accurate boundary of the entities, thus leading to unrecognized entity type. However, our RICON achieves detecting the correct span boundary as well as predicting golden type type (Symptom) of the entities according to the regularity "XX+病变" (XX+lesion). In the last three examples, both biaffine model and our RICON successfully detect the correct span boundary of the entities. For entity type prediction, the biaffine model assigns a wrong type (Disease) to these entities, but our RICON predicts types correctly as a result of it captures the regularity feature "XX+损害" (XX+damage) from "Symptom" type. To sum up, our RICON is also beneficial for domain datasets.