Abstract: Named entity recognition (NER) is the very first step in the linguistic processing of any new domain. It is currently a common process in BioNLP on English clinical text. However, it is still in its infancy in other major languages, as it is the case for Spanish. Presented under the umbrella of the PharmaCoNER shared task, this paper describes a very simple method for the annotation and normalization of pharmacological, chemical and, ultimately, biomedical named entities in clinical cases. The system developed for the shared task is based on limited knowledge, collected, structured and munged in a way that clearly outperforms scores obtained by similar dictionary-based systems for English in the past. Along with this recovering of the knowledge-based methods for NER in subdomains, the paper also highlights the key contribution of resource-based systems in the validation and consolidation of both the annotation guidelines and the human annotation practices. In this sense, some of the authors discoverings on the overall quality of human annotated datasets question the above-mentioned `official' results obtained by this system, that ranked second (0.91 F1-score) and first (0.916 F1-score), respectively, in the two PharmaCoNER subtasks.

Introduction: Named Entity Recognition (ner) is considered a necessary first step in the linguistic processing of any new domain, as it facilitates the development of applications showing co-occurrences of domain entities, cause-effect relations among them, and, eventually, it opens the (still to be reached) possibility of understanding full text content. On the other hand, Biomedical literature and, more specifically, clinical texts, show a number of features as regards ner that pose a challenge to NLP researchers BIBREF0: (1) the clinical discourse is characterized by being conceptually very dense; (2) the number of different classes for nes is greater than traditional classes used with, for instance, newswire text; (3) they show a high formal variability for nes (actually, it is rare to find entities in their “canonical form”); and, (4) this text type contains a great number of ortho-typographic errors, due mainly to time constraints when drafted. Many ways to approach ner for biomedical literature have been proposed, but they roughly fall into three main categories: rule-based, dictionary-based (sometimes called knowledge-based) and machine-learning based solutions. Traditionally, the first two approaches have been the choice before the availability of Human Annotated Datasets (had), albeit rule-based approaches require (usually hand-crafted) rules to identify terms in the text, while dictionary-based approaches tend to miss medical terms not mentioned in the system dictionary BIBREF1. Nonetheless, with the creation and distribution of had as well as the development and success of supervised machine learning methods, a plethora of data-driven approaches have emerged —from Hidden Markov Models (HMMs) BIBREF2, Support Vector Machines (SVMs) BIBREF3 and Conditional Random Fields (CRFs) BIBREF4, to, more recently, those founded on neural networks BIBREF5. This fact has had an impact on knowledge-based methods, demoting them to a second plane. Besides, this situation has been favoured by claims on the uselessness of gazetteers for ner in, for example, Genomic Medicine (GM), as it was suggested by BIBREF0 [p. 26]CohenandDemner-Fushman:2014: One of the findings of the first BioCreative shared task was the demonstration of the long-suspected fact that gazetteers are typically of little use in GM. Although one might think that this view could strictly refer to the subdomain of GM and to the past —BioCreative I was a shared task held back in 2004—, we can still find similar claims today, not only referred to rule-based and dictionary-based methods, but also to stochastic ones BIBREF5. In this paper, in spite of previous statements, we present a system that uses rule-based and dictionary-based methods combined (in a way we prefer to call resource-based). Our final goals in the paper are two-fold: on the one hand, to describe our system, developed for the PharmaCoNER shared task, dealing with the annotation of some of the nes in health records (namely, pharmacological, chemical and biomedical entities) using a revisited version of rule- and dictionary-based approaches; and, on the other hand, to give pause for thought about the quality of datasets (and, thus, the fairness) with which systems of this type are evaluated, and to highlight the key role of resource-based systems in the validation and consolidation of both the annotation guidelines and the human annotation practices. In section SECREF2, we describe our initial resources and explain how they were built, and try to address the issues posed by features (1) and (2) above. Section SECREF3 depicts the core of our system and the methods we have devised to deal with text features (3) and (4). Results obtained in PharmaCoNER by our system are presented in section SECREF4. Section SECREF5 details some of our errors, but, most importantly, focusses on the errors and inconsistencies found in the evaluation dataset, given that they may shed doubts on the scores obtained by any system in the competition. Finally, we present some concluding remarks in section SECREF6.

Question: How do they perform data augmentation?
Answer: They randomly sample sentences from Wikipedia that contains an object RC and add them to training data