M i DRED : An Annotated Corpus for Microbiome Knowledge Base Construction (2024)

Abstract: The interplay between microbiota and diseases has emerged as a significant area of research facilitated by the proliferation of cost-effective and precise sequencing technologies. To keep track of the many findings, domain experts manually review publications to extract reported microbe-disease associations and compile them into knowledge bases. However, manual curation efforts struggle to keep up with the pace of publications. Relation extraction has demonstrated remarkable success in other domains, yet the availability of datasets supporting such methods within the domain of microbiome research remains limited. To bridge this gap, we introduce the Microbe-Disease Relation Extraction Dataset (MiDRED); a human-annotated dataset containing 3,116 annotations of fine-grained relationships between microbes and diseases. We hope this dataset will help address the scarcity of data in this crucial domain and facilitate the development of advanced text-mining solutions to automate the creation and maintenance of microbiome knowledge bases.

## introduction microbiota play a pivotal role in human health in diverse environments such as the gut, skin, and oral cavity, influencing various physiological processes and disease mechanisms @xcite @xcite . the significance of microbiome research is underscored by its immense potential to unlock new understandings and treatments for various health conditions @xcite @xcite . for example, perturbations in gut microbiota composition, exemplified by fluctuations in bacteroidetes and firmicutes populations, have been linked to obesity and type 2 diabetes, respectively, providing valuable insights into the pathophysiology of these conditions @xcite . the growth of microbiome research introduces significant challenges in knowledge consolidation and utilization @xcite . current efforts often involve domain experts spending countless hours manually curating experimentally validated associations between diverse microbiota and diseases to form knowledge bases (kbs) @xcite @xcite . these kbs are invaluable for researchers and practitioners, providing a consolidated view of current findings, yet their maintenance is becoming unsustainable due to the rapid pace of publication. advanced text-mining methods designed to extract knowledge from biomedical texts are a well-established area of research @xcite @xcite @xcite @xcite . methods often leverage human-annotated data to train and validate a model's performance; however, robust datasets annotating microbe-disease associations are lacking. to address these challenges, we introduce midred; a comprehensive text-mining dataset designed to automate the construction and maintenance of microbiome kbs. midred consists of 3,116 annotated relationships between microbedisease pairs extracted from 1,655 scholarly articles. we specifically craft relation classes to align with classes used in major microbe-disease kbs to ensure midred's compatibility with existing databases. importantly, midred annotates negative instances (e.g., a "no relation" class) to mitigate positive bias from trained models @xcite . midred also includes span-level annotations of entities, which are crucial for training in named entity recognition (ner) and named entity normalization (nen) tasks. see the species-species interaction (ssi) dataset @xcite is a dataset that annotates binary associations between species of microbes. ssi does not provide human-annotated entities and relies on automated methods for ner. midred differs from ssi in entity types and the number of relation classes-in midred, we annotate four relation classes (see section 3.2 for more details), moving beyond binary associations. bacteria biotope (bb 2019) @xcite ) is an ner/re dataset featuring microbes, diseases, habitats, and locations. bb 2019 seeks to mine associations of microbes and environments (habitats) to better understand how microbes interact within various environments. midred, in contrast, focuses on how microbes relate to diseases more generally and offers a large number of annotated entities and relations. ## baseline experiments we explore the performance of popular nlp models using midred on named entity recognition (ner) and relation extraction (re) tasks to establish the baseline performance and highlight challenging areas for future development. ## results and discussion annotation challenges: numerous challenges were encountered when annotating microbes, diseases, and their associations. challenges with acronyms and abbreviations arose due to variations in naming conventions, which sometimes differed from standard classifications. relation types posed difficulties in accurately describing the links between microbe-disease pairs, particularly in cases involving numerical data or complex biological semantics. we record these and other challenges in appendix a.3 in hopes of improving future versions of midred and biomedical annotation efforts in general. ## conclusion microbiota, integral to human health and prevalent in various body environments like the gut, skin, and oral cavity, are at the forefront of promising research avenues that could revolutionize our understanding and treatment of numerous health conditions. however, the manual curation of microbiome knowledge bases, though invaluable, faces scalability challenges in keeping pace with the rapid influx of new research findings. in this paper, we introduce midred, a dataset that aims to bridge this gap by providing a resource to help automate the creation and maintenance of microbiome bases. midred can be used to train and validate stateof-the-art nlp models on various tasks such as named entity recognition, named entity normalization, bacteria-disease relationship extraction, and knowledge graph creation. we hope midred will unlock new applications and innovations within microbiome research.