SlowGuess/ACL_2020_OCR huggingface dataset:
P	R	F1
out-of-domain Word Embeddings
GloVe (newswire+Wiki+Web)	61.71	49.08	54.67
ELMo (newswire+Wiki)	67.66	47.41	55.75
Fine-tuned BERT (book+Wiki)	45.92	77.02	57.54
In-Domain Word Embeddings
GloVeOverflow	66.28	51.28	57.82
ELMoVerflow	74.44	68.71	71.46
Fine-tuned BERTOverflow	72.11	70.51	71.30
ELMo (Table 3), although it underperforms indomain ELMo (ELMoVerflow) by  $12.92\mathrm{F}_1$  score and in-domain BERt (BERTOverflow) by 12.76  $\mathbf{F}_1$  score (Table 2). Similarly, in-domain ELMo outperforms the out-of-domain fine-tuned BERT by  $10.67\mathrm{F}_1$  score on Github data (Table 8; more details in §5).
It is worth noting that, the performance improvements from contextual word embeddings are more pronounced on our software domain than on newswire and biomedical domains. Original ELMo and BERT outperform GloVe by 2.06 and 2.12 points in  $\mathrm{F}_1$  respectively on CoNLL 2003 NER task of newswire data (Peters et al., 2018; Devlin et al., 2019). For biomedical domain, indomain ELMo outperforms out-of-domain ELMo by only 1.33 points in  $\mathrm{F}_1$  on the BC2GM dataset (Sheikhshabbafghi et al., 2018).
We hypothesized that the performance gains from the in-domain contextual embeddings are largely aided by the model's ability to handle ambiguous and unseen tokens. The increase in performance is especially notable ( $41\% \rightarrow 70\%$  accuracy) for unseen tokens, which constitute  $38\%$  of the tokens inside gold entity spans in our dataset. This experiment also demonstrates that our annotated NER corpus provides an attractive test-bed for measuring the adaptability of different contextual word representations.
4.5 Evaluation of Auxiliary Systems
The domain-specific vectors produced by the Code Recognizer and the Entity Segmenter are also crucial for the overall performance of our SoftNER model. Table 4 shows an ablation study. Removing code recognizer vectors and entity segmenter vectors results in a drop of 2.19 and 3.69 in  $\mathrm{F_1}$  scores respectively. If we replace embedding-level attention with a simple concatenation of em
Table 3: Performance of fine-tuned BERT model, BiLSTM-CRF model with GloVe and ELMo embeddings on the dev set of our StackOverflow NER corpus. Contextualized word representations show a clear benefit when trained on the in-domain StackOverflow data.
P	R	F1
SoftNER	78.81	81.72	80.24
- Embedding Attention	75.83	79.09	77.43
- Code Recognizer	78.76	77.35	78.05
- Entity Segmenter	77.82	75.32	76.55
Table 4: Ablation study of SoftNER on the dev set of StackOverflow NER corpus.
P	R	F1
Token Frequency	33.33	2.25	4.22
Most Frequent Label	82.21	58.59	68.42
Our Code Recognition Model	78.43	83.33	80.80
- Character ngram LMs	64.13	84.51	72.90
- Word ngram LMs	67.98	72.96	70.38
- FastText Embeddings	76.12	81.69	78.81
Table 5: Evaluation results and feature ablation of our code recognition model on SOLEXICON test set of 1000 manually labeled unique tokens, which are sampled from the train set of StackOverflow NER corpus.
beddings, the performance also drop by  $2.81\mathrm{F}_1$ . In addition, we evaluate the effectiveness of our two domain-specific auxiliary systems on their respective tasks.
Code Recognition: Table 5 compares the performance of our code recognition model with other baselines on the SLEXICON test set (§4.1), which consists of 1,000 random words from the train set of StackOverflow NER corpus classified as either a code or a non-code token. The baselines include: (i) a Most Frequent Label baseline, which assigns the most frequent label according to the human annotation in SOLEXICON train set; and (ii) a frequency baseline, which learns a threshold over token frequency in the train set of StackOverflow NER corpus using a decision tree classifier. Our model outperforms both baselines in terms of  $\mathrm{F}_1$  score. Although the most frequent label baseline achieves better precision than our model, it performs poorly on unseen tokens resulting in a large drop in recall and  $\mathrm{F}_1$  score. The ablation experiments show that the FastText word embeddings along with the character and word-level features are crucial for the code recognition model.
Entity Segmentation: Table 6 shows the performance of our segmentation model on the dev set of our StackOverflow corpus, where the entity tags are replaced by an I-ENTITY tag. Our model achieves an  $\mathrm{F}_1$  score of 88.09 and with 90.41% precision and 85.89% recall. Incorporating word frequency and code markdown feature increases the  $\mathrm{F}_1$  score by 1.57 and 2.66 points respectively. The low 10.5  $\mathrm{F}_1$  score of Stanford NER
P	R	F1
Stanford NER Tagger	63.02	5.74	10.52
Our Entity Segmentation Model	90.41	85.89	88.09
- Word Frequency	88.32	84.79	86.52
- Code Markdown	86.23	84.64	85.43
Table 6: Evaluation of our segmentation model on the dev set of the StackOverflow NER corpus.
tagger (Manning et al., 2014), which is trained on newswire text, demonstrates the importance of domain-specific tools for the software engineering domain.
4.6 Error Analysis
Based on our manual inspection, the incorrect predictions made by NER systems on StackOverflow data can be largely classified into the following two categories (see examples in Table 7):
- Segmentation Mismatch refers to the cases where model predicts the boundary of entities incorrectly. Our SoftNER model reduces such segmentation errors by  $89.36\%$  compared to the fine-tuned BERTOverflow baseline.
- Entity-Type Mismatch refers to the errors where a code entity (e.g., names of variables) is predicted as a non-code entity (e.g., names of devices), and vice-versa. Our SoftNER model reduces such entity type errors by  $13.54\%$  compared to the fine-tuned BERTOverflow baseline.
As illustrated in Figure 3, our SoftNER model reduced the errors in both categories by incorporating the auxiliary outputs from segmenter and code recognizer model.
5 Domain Adaptation to GitHub data
To understand the domain adaptability of our StackOverflow based SoftNER, we evaluate its performance on readme files and issue reports from 143 randomly sampled repositories in the GitHub dump (Gousios and Spinellis, 2012). We also trained ELMo embeddings (ELMoGithub) on 4 million sentences from randomly sampled 5,000 GitHub repositories.
Table 8 shows that the performance of our SoftNER model using StackOverflow ELMo embeddings is similar to the top performing BiLSTM-CRF model using GitHub ELMo embeddings with a difference of only 1.61 points in  $\mathrm{F}_1$ . We also did not observe any significant gain after adding
Segmentation
Mismatch	Gold:	Operating System Version
Windows Server 2012 R2.
Predicted:	Operating System Version
Windows Server 2012 R2.
Entity-Type
Mismatch	Library
Gold:	I used ShareKit which was a breeze to add.
Application
Predicted:	I used ShareKit which was a breeze to add.
Figure 3: Comparison of errors made by the fine-tuned BERTOverflow baseline and our SoftNER model on the dev set of the StackOverflow NER corpus. In the heatmap, darker cell color corresponds to higher error counts. Our SoftNER model reduces errors in all the categories.
Table 7: Representative examples of system errors.
P	R	F1
Feature-Based CRF	43.16	35.71	39.09
BiLSTM-CRF (ELMoGitHub)	64.53	60.96	62.69
Attentive BiLSTM-CRF (ELMoVerflow)	62.05	59.20	60.59
Attentive BiLSTM-CRF (ELMoGitHub)	63.29	60.89	62.07
Fine-tuned out-of-domain BERT	56.59	48.13	52.02
Fine-tuned BERTOverflow	61.71	58.75	60.19
SoftNER (BERTOverflow)	61.92	60.26	61.08
Table 8: Evaluation on the GitHub NER dataset of README files and issue posts. All the models are trained on our StackOverflow NER corpus. Our SoftNER model performs close to BiLSTM-CRF model trained on the GitHub ELMo embeddings.
the code recognizer and segmenter vectors to the Github ELMo embeddings. We think one likely explanation is that GitHub data contains less code-related tokens when compared to StackOverflow. The percentage of code-related entity tokens is  $63.20\%$  in GitHub and  $77.21\%$  in StackOverflow. Overall, we observe a drop of our SoftNER tagger from  $79.10~\mathrm{F_1}$  on StackOverflow (Table 2) to  $61.08~\mathrm{F_1}$  on GitHub data (Table 8) in  $\mathbf{F}_1$  due to domain mismatch. However, we believe that our NER tagger still achieves sufficient performance to be useful for applications on GitHub.<sup>7</sup> We leave investigation of semi-supervised learning and other domain adaptation approaches for future work.
7As a reference, the state-of-the-art performance for 10-class Twitter NER is  $70.69\mathrm{F}_1$  (Zhang et al., 2018).
The CoNLL 2003 dataset (Sang and De Meulder, 2003) is a widely used benchmark for named entity recognition, which contains annotated newswire text from the Reuters RCV1 corpus. State-of-the-art approaches on this dataset (Baevski et al., 2019) use a bidirectional LSTM (Lample et al., 2016; Ma and Hovy, 2016) with conditional random field (Collobert et al., 2011) and contextualized word representations (McCann et al., 2017; Peters et al., 2018; Devlin et al., 2019).
Named entity recognition has been explored for new domains and languages, such as social media (Finin et al., 2010; Ritter et al., 2011; Plank et al., 2014; Derczynski et al., 2015; Limsopatham and Collier, 2016; Aguilar et al., 2017), biomedical texts (Collier and Kim, 2004; Greenberg et al., 2018; Kulkarni et al., 2018), multilingual texts (Benajiba et al., 2008; Xie et al., 2018) and code-switched corpora (Aguilar et al., 2018; Ball and Garrette, 2018). Various methods have been investigated for handling rare entities, for example incorporating external context (Long et al., 2017) or approaches that make use of distant supervision (Choi et al., 2018; Yang et al., 2018; Onoe and Durrett, 2019).
There has been relatively little prior work on named entity recognition in the software engineering domain. Ye et al. (2016) annotated 4,646 sentences from StackOverflow with five named entity types (Programming Language, Platform, API, Tool-Library-Framework and Software Standard). The authors used a traditional feature-based CRF to recognize these entities. In contrast, we present a much larger annotated corpus consisting of 15,372 sentences labeled with 20 fine-grained entity types. We also develop a novel attention based neural NER model to extract those fine-grained entities.
In this work, we investigated the task of named entity recognition in the social computer programming domain. We developed a new NER corpus of 15,372 sentences from StackOverflow and 6,510 sentences from GitHub annotated with 20 fine-grained named entities. We demonstrate that this new corpus is an ideal benchmark dataset for contextual word representations, as there are many challenging ambiguities that often require
long-distance context to resolve. We also proposed a novel attention based model, named SoftNER, that outperforms the state-of-the-art NER models on this dataset. Furthermore, we investigated the important sub-task of code recognition. Our code recognition model captures additional spelling information beyond then contextual word representations and consistently helps to improve the NER performance. We believe our corpus, StackOverflow-specific BERT embeddings and named entity tagger will be useful for various language-and-code tasks, such as code retrieval, software knowledge base extraction and automated question-answering.
We thank anonymous reviewers for their thoughtful comments. We also thank NVIDIA, Google, and Ohio Supercomputer Center (Center, 2012) for providing GPU/TPU computing resources; Wuwei Lan for kindly helping to train in-domain BERT on StackOverflow data; Sydney Lee, Rita Tong, Lillian Chow, and Raleigh Potluri for help with data annotation. This research is supported in part by the NSF awards IIS-1822754 and IIS-1845670, ODNI and IARPA via the BETTER program contract 19051600004, ARO and DARPA via the SocialSim program contract W911NF-17-C-0095, Criteo Faculty Research Award to Wei Xu, and Amazon Faculty Research Award to Alan Ritter. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of NSF, ODNI, IARPA, ARO, DARPA or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein.
Gustavo Aguilar, Fahad AlGhamdi, Victor Soto, Mona Diab, Julia Hirschberg, and Thamar Solorio. 2018. Named Entity Recognition on Code-Switched Data: Overview of the CALCS 2018 Shared Task. In Proceedings of the Third Workshop on Computational Approaches to Linguistic Code-Switching.
Gustavo Aguilar, Suraj Maharjan, Adrian Pastor López-Monroy, and Thamar Solorio. 2017. A Multi-task Approach for Named Entity Recognition in Social Media Data. In Proceedings of the 3rd Workshop on Noisy User-generated Text (WNUT).
David Lo Thamar Solorio Amin Alipour Amirreza Shirani, Bowen Xu. 2019. Question Relatedness on Stack Overflow: The Task, Dataset, and Corpus-inspired Models. In Proceedings of the AAAI Reasoning for Complex Question Answering Workshop.
Alexei Baevski, Sergey Edunov, Yinhan Liu, Luke Zettlemoyer, and Michael Auli. 2019. Cloze-driven pretraining of self-attention networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics.
Kelsey Ball and Dan Garrette. 2018. Part-of-Speech Tagging for Code-Switched, Transliterated Texts without Explicit Language Identification. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP).
Yassine Benajiba, Mona Diab, and Paolo Rosso. 2008. Arabic Named Entity Recognition using Optimized Feature Sets. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP).
Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural Language Processing with Python. O'Reilly Media Inc.
Ohio Supercomputer Center. 2012. Oakley supercomputer.
Eunsol Choi, Omer Levy, Yejin Choi, and Luke Zettlemoyer. 2018. Ultra-Fine Entity Typing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL).
Jacob Cohen. 1960. A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement.
Nigel Collier and Jin-Dong Kim. 2004. Introduction to the Bio-entity Recognition Task at JNLPBA. In Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications (NLPBA/BioNLP).
Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural Language Processing (almost) from Scratch. Journal of Machine Learning Research (JMLR).
Xiang Dai, Sarvnaz Karimi, Ben Hachey, and Cecile Paris. 2019. Using Similarity Measures to Select Pretraining Data for NER. arXiv preprint arXiv:1904.00585.
Leon Derczynski, Diana Maynard, Giuseppe Rizzo, Marieke Van Erp, Genevieve Gorrell, Raphael Troncy, Johann Petrak, and Kalina Bontcheva. 2015. Analysis of Named Entity Recognition and Linking for Tweets. Information Processing & Management.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL).
Tim Finin, Will Murnane, Anand Karandikar, Nicholas Keller, Justin Martineau, and Mark Dredze. 2010. Annotating Named Entities in Twitter Data with Crowdsourcing. In Proceedings of the Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk.
Kevin Gimpel, Nathan Schneider, Brendan O'Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah A. Smith. 2011. Part-of-speech Tagging for Twitter: Annotation, Features, and Experiments. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT).
John M Giorgi and Gary Bader. 2018. Transfer Learning for Biomedical Named Entity Recognition with Neural Networks. bioRxiv.
G. Gousios and D. Spinellis. 2012. GHTorrent: Github's Data from a Firehose. In Proceedings of the 9th IEEE Conference on Mining Software Repositories (MSR).
Nathan Greenberg, Trapit Bansal, Patrick Verga, and Andrew McCallum. 2018. Marginal Likelihood Training of BiLSTM-CRF for Biomedical Named Entity Recognition from Disjoint Label Sets. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP).
Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, and Luke Zettlemoyer. 2016. Summarizing Source Code using a Neural Attention Model. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL).
Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Hérve Jégou, and Tomas Mikolov. 2016. FastText.zip: Compressing Text Classification Models. arXiv preprint arXiv:1612.03651.
Chaitanya Kulkarni, Wei Xu, Alan Ritter, and Raghu Machiraju. 2018. An Annotated Corpus for Machine Reading of Instructions in Wet Lab Protocols. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT).
Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural Architectures for Named Entity Recognition. In Proceedings of the 2016 Conference of the North
American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT).
Nut Limsopatham and Nigel Collier. 2016. Bidirectional LSTM for Named Entity Recognition in Twitter Messages. In Proceedings of 2016 the Workshop on Noisy User-generated Text (WNUT).
Dan Liu, Wei Lin, Shiliang Zhang, Si Wei, and Hui Jiang. 2016. Neural Networks Models for Entity Discovery and Linking. arXiv preprint arXiv:1611.03558.
Teng Long, Emmanuel Bengio, Ryan Lowe, Jackie Chi Kit Cheung, and Doina Precup. 2017. World Knowledge for Reading Comprehension: Rare Entity Prediction with Hierarchical LSTMs Using External Descriptions. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP).
Xuezhe Ma and Eduard Hovy. 2016. End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL).
Mounica Maddela and Wei Xu. 2018. A Word-Complexity Lexicon and A Neural Readability Ranking Model for Lexical Simplification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP).
Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. 2014. The Stanford CoreNLP Natural Language Processing Toolkit. In Proceedings of the 2014 Association for Computational Linguistics System Demonstrations (ACL).
Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017. Learned in Translation: Contextualized Word Vectors. In Proceedings of the 2017 Conference on Neural Information Processing Systems (NeurIPS).
Dana Movshovitz-Attias and William W Cohen. 2015. KB-LDA: Jointly Learning a Knowledge Base of Hierarchy, Relations, and Facts. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-IJCNLP).
Courtney Naples, Matthew Gormley, and Benjamin Van Durme. 2012. Annotated Gigaword. In Proceedings of the Joint Workshop on Automatic Knowledge Base Construction and Web-Scale Knowledge Extraction.
Yasumasa Onoe and Greg Durrett. 2019. Learning to Denoise Distantly-Labeled Data for Entity Typing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NACL).
Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global Vectors for Word Representation. In Proceedings of the Empirical Methods in Natural Language Processing (EMNLP).
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep Contextualized Word Representations. In Proceedings of the of Conference of the North American Chapter of the Association for Computational Linguistics (NAACL).
Barbara Plank, Dirk Hovy, and Anders Søgaard. 2014. Learning part-of-speech taggers with inter-annotator agreement loss. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics (EACL).
Chris Quirk, Raymond Mooney, and Michel Galley. 2015. Language to Code: Learning Semantic Parsers for If-This-Then-That Recipes. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-IJCNLP).
Sujith Ravi, Bo Pang, Vibhor Rastogi, and Ravi Kumar. 2014. Great Question! Question Quality in Community Q&A. In Eighth International AAAI Conference on Weblogs and Social Media.
Alan Ritter, Sam Clark, Mausam, and Oren Etzioni. 2011. Named Entity Recognition in Tweets: An Experimental Study. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing (EMNLP).
Erik F Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL.
Golnar Sheikhshabbafghi, Inanc Birol, and Anoop Sarkar. 2018. In-domain Context-aware Token Embeddings Improve Biomedical Named Entity Recognition. In Proceedings of the Ninth International Workshop on Health Text Mining and Information Analysis.
Avirup Sil, Gourab Kundu, Radu Florian, and Wael Hamza. 2017. Neural Cross-Linguual Entity Linking. In Proceedings of the 30th AAAI Conference on Artificial Intelligence.
Pontus Stenetorp, Sampo Pyysalo, Goran Topic, Tomoko Ohta, Sophia Ananiadou, and Jun'ichi Tsujii. 2012. brat: a Web-based Tool for NLP-Assisted Text Annotation. In Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics (EACL).
Minghao Wu, Fei Liu, and Trevor Cohn. 2018. Evaluating the Utility of Hand-crafted Features in Sequence Labelling. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP).
Jiateng Xie, Zhilin Yang, Graham Neubig, Noah A Smith, and Jaime Carbonell. 2018. Neural Cross-Lingual Named Entity Recognition with Minimal Resources. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP).
Yaosheng Yang, Wenliang Chen, Zhenghua Li, Zhengqiu He, and Min Zhang. 2018. Distantly Supervised NER with Partial Annotation Learning and Reinforcement Learning. In Proceedings of the 27th International Conference on Computational Linguistics (COLING).
Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical Attention Networks for Document Classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL).
Ziyu Yao, Jayavardhan Reddy Peddamail, and Huan Sun. 2019. CoaCor: Code Annotation for Code Retrieval with Reinforcement Learning. In Proceedings of the World Wide Web Conference (WWW).
Deheng Ye, Zhenchang Xing, Chee Yong Foo, Zi Qun Ang, Jing Li, and Nachiket Kapre. 2016. Software-specific Named Entity Recognition in Software Engineering Social Content. In Proceedings of the 2016 IEEE International Conference on Software Analysis, Evolution, and Reengineering (SANER).
Pengcheng Yin and Graham Neubig. 2018. TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP).
Qi Zhang, Jinlan Fu, Xiaoyu Liu, and Xuanjing Huang. 2018. Adaptive Co-attention Network for Named Entity Recognition in Tweets. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence.
A Feature-Based CRF Baseline