Towards Automatic Curation of Antibiotic Resistance Genes via Statement Extraction from Scientific Papers: A Benchmark Dataset and Models (2022)

Abstract: Antibiotic resistance has become a growing worldwide concern as new resistance mechanisms are emerging and spreading globally, and thus detecting and collecting the cause – Antibiotic Resistance Genes (ARGs), have been more critical than ever. In this work, we aim to automate the curation of ARGs by extracting ARG-related assertive statements from scientific papers. To support the research towards this direction, we build SciARG, a new benchmark dataset containing 2,000 manually annotated statements as the evaluation set and 12,516 silver-standard training statements that are automatically created from scientific papers by a set of rules. To set up the baseline performance on SciARG, we exploit three state-of-the-art neural architectures based on pre-trained language models and prompt tuning, and further ensemble them to attain the highest 77.0% F-score. To the best of our knowledge, we are the first to leverage natural language processing techniques to curate all validated ARGs from scientific papers. Both the code and data are publicly available at https://github.com/VT-NLP/SciARG .

## introduction antibiotic resistance (ar), the ability of bacteria to survive and propagate in the presence of antibiotics, is a prevalent phenomenon worldwide and poses a serious health threat to humans and animals. automatically detecting the antibiotic resistance genes (args-the root cause of ar) in clinical and natural environments has been critical for mitigating the spread of ar. however, though the research on args has grown exponentially over the past 10-15 years, existing arg databases, such as card @xcite , ardb @xcite , argo @xcite , and argminer @xcite , only contain a fraction of args that have been discovered and validated by researchers, making it difficult to fully keep track of the research on args. statement 1: gram-negative enterobacteriaceae with resistance to carbapenem conferred by new delhi metallo-beta-lactamase 1 (blandm-1) are potentially a major global health problem. ## approach to set up the baseline performance on sciarg, we exploit three supervised approaches. ## remaining challenges to understand the remaining challenges of sciarg, we randomly sample 100 prediction errors of the ensembling approach from the development set, and summarize the following three key remaining challenges. ## conclusion and future work in this work, we present the first computational framework that aims to automatically curate args by extracting arg-related assertative statements from scientific papers in pubmed. to support the research, we introduce sciarg, a dataset that contains 2,000 manually annotated statements as the test set and 12,516 silver-standard training statements that are automatically created from scientific papers by a set of rules. we also present extensive empirical results by comparing various state-ofthe-art neural architectures based on pre-trained language models for statement classification, and demonstrate that there is still a large room to improve based on the current highest 77% f-score on sciarg. considering the remaining challenges that we have discussed, there are multiple future directions: (1) developing more advanced frameworks that incorporate domain-specific knowledge from external resources or knowledge bases to better interpret the statements; (2) learning contextual features of target genes from more broad context, such as the paragraph, chapter or the whole document; (3) leveraging self-training or co-training framework to take advantage of the large-scale unlabeled corpus from pubmed to enrich the training samples.