C hem NER : Fine-Grained Chemistry Named Entity Recognition with Ontology-Guided Distant Supervision (2021)

Abstract: Scientific literature analysis needs fine-grained named entity recognition (NER) to provide a wide range of information for scientific discovery. For example, chemistry research needs to study dozens to hundreds of distinct, fine-grained entity types, making consistent and accurate annotation difficult even for crowds of domain experts. On the other hand, domain-specific ontologies and knowledge bases (KBs) can be easily accessed, constructed, or integrated, which makes distant supervision realistic for fine-grained chemistry NER. In distant supervision, training labels are generated by matching mentions in a document with the concepts in the knowledge bases (KBs). However, this kind of KB-matching suffers from two major challenges: incomplete annotation and noisy annotation. We propose ChemNER, an ontology-guided, distantly-supervised method for fine-grained chemistry NER to tackle these challenges. It leverages the chemistry type ontology structure to generate distant labels with novel methods of flexible KB-matching and ontology-guided multi-type disambiguation. It significantly improves the distant label generation for the subsequent sequence labeling model training. We also provide an expert-labeled, chemistry NER dataset with 62 fine-grained chemistry types (e.g., chemical compounds and chemical reactions). Experimental results show that ChemNER is highly effective, outperforming substantially the state-of-the-art NER methods (with .25 absolute F1 score improvement).

## introduction named entity recognition (ner) is a fundamental step in scientific literature analysis to build ai-driven systems for molecular discovery, synthetic strategy designing, and manufacturing @xcite @xcite @xcite . it aims to locate and classify entity mentions (e.g., "suzuki-miyaura cross-coupling reactions") from unstructured text into pre-defined categories (e.g., "coupling reactions"). in the chemistry domain, previous ner studies are mostly focused on one coarse-grained entity type (i.e., chemicals) @xcite @xcite and rely on large amounts of manuallyannotated data for training deep learning models @xcite @xcite @xcite . in real-world applications, it is important to recognize chemistry entities on diverse and finegrained types (e.g., "inorganic phophorus compounds", "coupling reactions" and "catalysts") to provide a wide range of information for scientific discovery. it will need dozens to hundreds of distinct types, making consistent and accurate annotation difficult even for domain experts. on the other hand, the domain-specific ontologies and knowledge bases (kbs) can be easily accessed, constructed, or integrated, which makes distant supervision realistic for fine-grained chemistry ner. still, challenges exist for correctly recognizing the entity boundaries and accurately typing entities with distant supervision. in distant supervision, training labels are generated by matching the mentions in a document with the concepts in the knowledge bases (kbs). however, this kind of kb-matching suffers from two major challenges: (1) incomplete annotation where a mention in a document can be matched only partially or missed completely due to an incomplete coverage of the kbs (figure 1a ), and (2) noisy annotation where a mention can be erroneously matched due to the potential matching of multiple entity types in the kbs (figure 1b ). due to the complex name structures (e.g., nested naming structures and long chemical formulas) of chemical entities, these challenges lead to severe low-precision and low-recall for finegrained chemistry ner with distant supervision. several studies have attempted to address the incomplete annotation problem in distantlysupervised ner. for example, autoner @xcite introduces an "unknown" type that can be skipped during training to reduce the effect of false negative labeling with distant supervision. bond @xcite leverages the power of pre-trained language models and a self-training approach to iteratively incorporate more training labels and improve the ner performance. however, previous methods assume a high precision and reasonable coverage of kb-matching for distant label generation. for example, the kb-matching on the conll03 dataset @xcite reported over 80% on precision and over 60% on recall. these methods do not work well with fine-grained chemistry ner that has severe low precision and low recall with kb-matching. previous studies also largely ignore the noisy annotation problem by simply discarding those multi-labels during the kbmatching process @xcite . however, the noisy labels cannot be simply ignored for the chemistry entities because they consist of a large portion of distant training labels. we observe that more than 60% of the entities have multiple labels during kb-matching in the chemistry domain. we propose chemner, an ontology-guided, distantly-supervised ner method for fine-grained chemistry ner. taking an input corpus, a chemistry type ontology and associated entity dictionaries collected from the kbs, we develop a novel flexible kb-matching method with tf-idf-based majority voting to resolve the incomplete annota-tion problem. then we develop a novel ontologyguided multi-type disambiguation method to resolve the noisy annotation problem. taking the output from the above two steps as distant supervision, we further train a sequence labeling model to cover additional entities. chemner significantly improves the distant label generation for the subsequent ner model training. we also provide an expert-labeled, chemistry ner dataset with 62 finegrained chemistry types (e.g., chemical compounds and chemical reactions). experimental results show that chemner is highly effective, achieving substantially better performance (with .25 absolute f1 score improvement) compared with the state-ofthe-art ner methods. we have released our data and code to benefit future studies foot_0 . autoner @xcite proposes a "tieor-break" tagging scheme to leverage distant supervision from entity dictionaries. compared with the traditional "bioes" tagging scheme, the "tie-orbreak" tagging scheme introduces an "unknown" type that can be skipped during training to reduce the effect of false negative labeling brought by the incomplete kb-matching. however, autophrase often misses low-frequency phrases for the "unknown" entity generation using a phrase mining method autophrase @xcite . positive and unlabeled learning (pu-learning) is used in distantly-supervised ner to provide an unbiased and consistent estimator of the objective function @xcite . however, there are two limitations in using pu-learning for distantly-supervised ner. first, pu-learning uses the prior distribution for each entity type, a parameter that is estimated from an existing human-annotated test set that is not always available for new entity types. second, the performance of pu-learning is highly sensitive to the class-imbalance rate for each entity type, a knowledge bases s1: [methyl-14c]s-dthd was synthesized by rapid methylation of ... s2: ... suzuki-miyaura cross-coupling reactions were carried out … s3: although it was necessary to employ a stoichiometric quantity of palladium , it is noteworthy that the cross-coupling proceeded in the presence of a wide array of functional groups. s4: … can undergo a transmetalation with either bba or the rapidly forming boronic acid … entity span detection flexible kb-matching s1: [methyl-14c]s-dthd was synthesized by rapid methylation of ... s2: ..., suzuki-miyaura cross-coupling reactions were carried out ... tf-idf scores organic compounds organic polymers biomolecules … methyl 0.0177 0.0139 0.0010 … thd 0.0256 0.0115 0.0417 organic compounds, organic polymers ontology-guided multi-type disambiguation s3: although it was necessary to employ a stoichiometric quantity of palladium , it is noteworthy that the cross-coupling proceeded in the presence of a wide array of functional groups. chemical reactions catalysts transition metals … sequence labeling models bilsmt-crf, roberta, chemberta, … s4: … can undergo a transmetalation with either bba or the rapidly forming boronic acid … … candidate types freq. catalysts 18,707 transition_metals 9,618 context type chemical elements candidate types … … catalysts, transition metals functional groups catalysis organic reactions … … coupling reactions chemistry coupling reactions organic reactions oxoacids ??? [not in kb] => oxoacids "either ... or …" pattern learned by sequence labeling model organometallic chemistry input corpus coupling reactions figure 2: the overall framework of chemner. it includes a distant label generation (entity span detection, flexible kb-matching, and ontology-guided multi-type disambiguation) and a sequence labeling model training. other related tasks. one similar task to finegrained ner is entity linking @xcite @xcite ) that maps a candidate entity in the text to a concept identifier in the knowledge bases. however, entity linking cannot deal with new entities that do not exist in the background knowledge bases. another similar task is fine-grained entity typing (fet) @xcite @xcite @xcite that has been extensively studied in the general domain. fet aims at classifying an entity mention into a wide range of entity types by disambiguating the pre-identified entity mentions into a set of candidate entity types. it is formulated as a multi-class, multi-label classification problem and does not assume type exclusiveness. the fine-grained ner task targets both entity boundary detection and entity type recognition and assumes each entity to be tagged with only one type in a given context. in this study, we focus on the finegrained ner task in the chemistry domain. ## the chemner framework we propose chemner, an ontology-guided distantly-supervised ner method for fine-grained chemistry ner (figure 2 ). it includes distant label generation (entity span detection, flexible kbmatching, and ontology-guided multi-type disambiguation) and sequence labeling model training. ## conclusions and future work we propose chemner, an ontology-guided, distantly-supervised method for fine-grained chemistry ner. it leverages the chemistry type ontology structure to generate distant labels with novel methods of flexible kb-matching and ontology-guided multi-type disambiguation. we also provide an expert labeled, chemistry ner dataset with 62 finegrained chemistry types (e.g., chemical compounds and chemical reactions). experimental results show that chemner is highly effective, outperforming substantially the state-of-the-art ner methods on fine-grained chemistry ner. although achieving great performance, there is still large room for improvement of chemner. in the future, we plan to further refine and enrich the type ontology and incorporate more information in the dictionaries (e.g., chemical structures in the kbs) for a better ner performance. we also plan to apply our finegrained ner method to other scientific domains.