Chem- FINESE : Validating Fine-Grained Few-shot Entity Extraction through Text Reconstruction (2024)

Abstract: Fine-grained few-shot entity extraction in the chemical domain faces two unique challenges. First, compared with entity extraction tasks in the general domain, sentences from chemical papers usually contain more entities. Moreover, entity extraction models usually have difficulty extracting entities of long-tailed types. In this paper, we propose Chem-FINESE, a novel sequence-to-sequence (seq2seq) based few-shot entity extraction approach, to address these two challenges. Our Chem-FINESE has two components: a seq2seq entity extractor to extract named entities from the input sentence and a seq2seq self-validation module to reconstruct the original input sentence from extracted entities. Inspired by the fact that a good entity extraction system needs to extract entities faithfully, our new self-validation module leverages entity extraction results to reconstruct the original input sentence. Besides, we design a new contrastive loss to reduce excessive copying during the extraction process. Finally, we release ChemNER+, a new fine-grained chemical entity extraction dataset that is annotated by domain experts with the ChemNER schema. Experiments in few-shot settings with both ChemNER+ and CHEMET datasets show that our newly proposed framework has contributed up to 8.26% and 6.84% absolute F1-score gains respectively.

## introduction millions of scientific papers are published annually foot_1 , resulting in an information overload @xcite . due to such an explosion of research directions, it is impossible for scientists to fully explore the landscape due to sentence reconstructed from name tagging results figure 1 : comparison of sentence reconstruction results from ground truth and inboxbart @xcite . we highlight complete correct, missed entity, and partially correct prediction with different color. the limited reading ability of humans. therefore, information extraction, especially entity extraction of fine-grained scientific entity types, becomes a crucial step to automatically catch up with the newest research findings in the chemical domain. despite such a pressing need, fine-grained entity extraction in the chemical domain presents three distinctive and non-trivial challenges. first, there are very few publicly available benchmarks with high-quality annotations on fine-grained chemical entity types. for example, chemner @xcite developed the first fine-grained chemistry entity extraction dataset. however, their dataset is not released publicly. to address this issue, we collaborate with domain experts to annotate chem-ner+, a new chemical entity extraction dataset based on the chemner ontology. besides, we construct another new fine-grained entity extraction dataset based on an existing entity typing dataset chemet @xcite . in addition, current entity extraction systems in few-shot settings face two main problems: missing mentions and incorrect long-tail predictions. one primary reason for missing mentions is that the sentences in scientific papers typically cover more entities than sentences in the general domain. for example, there are 3.1 entities per sentence in our chemner+ dataset, which is much higher than the 1.5 entities in the general domain dataset conll2003 @xcite . as a result, it is more difficult for entity extraction models to cover all mentions in the input sentences. as shown in figure 1 , since the input has already included four chemical entities, in-boxbart model @xcite completely misses the entity "room temperature". furthermore, entity distributions in the chemical domain are highly imbalanced. as shown in figure 2 , we observe that the entity type distributions of chemner+ and chemet exhibit similar long-tail patterns. in few-shot settings, entities with long-tail types are extremely difficult to extract due to insufficient training examples. for example, as shown in figure 1 , inboxbart mistakenly predicts the entity "aryl sulfamates" as catalyst, because its type has a frequency forty times lower than the predicted type (i.e., 4 vs 136). moreover, the diverse representation nature of chemical entities-such as trade names, trivial names, and semi-systematic names (e.g., thf, iprmgcl, 8-phenyl ring)-makes it even harder for models to generalize on these long-tail entities. ## conclusion and future work in this paper, we introduce a novel framework for chemical fine-grained entity extraction. specifi-cally, we target two unique challenges for few-shot fine-grained scientific entity extraction: mention coverage and long-tail entity extraction. we build a new self-validation module to automatically proofread the entity extraction results and a novel decoder contrastive loss to reduce excessive copying. experimental results show that our proposed model achieves significant performance gains on two datasets: chemner+ and chemet. in the future, we plan to explore incorporating an external knowledge base to further improve the model's performance. specifically, we plan to inject type definition into the representation to facilitate the entity extraction procedure. we will also continue exploring the use of constraint decoding to further improve entity extraction quality.