Linking a Hypothesis Network From the Domain of Invasion Biology to a Corpus of Scientific Abstracts: The INAS Dataset (2022)

Abstract: We investigate the problem of identifying the major hypothesis that is addressed in a scientific paper. To this end, we present a dataset from the domain of invasion biology that organizes a set of 954 papers into a network of fine-grained domain-specific categories of hypotheses. We carry out experiments on classifying abstracts according to these categories and present a pilot study on annotating hypothesis statements within the text. We find that hypothesis statements in our dataset are complex, varied and more or less explicit, and, importantly, spread over the whole abstract. Experiments with BERT-based classifiers show that these models are able to classify complex hypothesis statements to some extent, without being trained on sentence-level text span annotations.

## introduction in many disciplines of science, researchers need to develop specific hypotheses that make it possible to confront general scientific claims with empirical evidence @xcite . for instance, studies in invasion biology, a sub-discipline of biodiversity research, investigate why certain species can establish in new ecosystems and typically formulate hypotheses specific to the species or the forms of invasion success they address (see figure 2 ). it is essential for a researcher to be aware of the existing hypotheses in these fields, but, to date, structured information on claims and hypotheses investigated in a field is often hardly available. in some cases, though, valuable resources and overviews are compiled manually by domain experts as, for instance, @xcite 's hierarchical network of hypotheses synthesizing research in the field of invasion biology. in this paper, we propose to leverage this resource as a new dataset for domain-specific information extraction from scientific publications and explore the potential of stateof-the-art off-the-shelf nlp models for automatic hypothesis identification. extracting domain-specific information on hypotheses from scientific publications is still a considerable challenge for state-of-the-art approaches in nlp and ie. research on ie for the biodiversity domain provides many annotated datasets and models with domain-specific labeling schemes for named entities and relations -e.g., species, locations and habitats @xcite -but does not account for more complex entities like claims, research questions or hypotheses. work on argumentation mining for scientific texts @xcite annotate argumentative spans of texts, including claims, but do not link them to domain-specific knowledge. however, the lack of domain-specific categories is a major gap in existing search repositories for biodiversity researchers, as shown by @xcite . in this work, we perform initial studies on the automatic extraction of information on hypotheses investigated in scientific publications. we compile a corpus of scientific abstracts, based on metadata in @xcite 's hypothesis network for invasion biology. we release the resulting inas dataset that links 954 scientific papers (with abstracts and titles) to nodes in a hypothesis network. similar to datasets in relation extraction @xcite , the inas dataset is weakly labeled, as the hypotheses are linked to the abstract as a whole, and not annotated in terms of text spans. we present a pilot analysis on hypothesis statements within the texts and find that they are complex, varied and spread over the whole abstract, challenging existing labeling schemes in ie. we carry out experiments on labeling abstracts with bert-based classifiers and show that these models are able to detect fine-grained hypothesis categories to some extent, without being trained on text span annotations. this shows that domain-specific resources on hypotheses provide a valuable starting point for this complex ie task, and points to some challenges for future research on automatic hypothesis extraction. ## related work our work combines ideas from named entity recognition (ner) and relation extraction (re), which typically targets domain-specific tagging schemes, with ideas of domain-general mining of claims, which aims at discovering complex statements of claims in text. we will briefly discuss related work from these areas in the following. ## the inas dataset we now introduce the inas dataset foot_0 , which is based on an existing resource that organizes papers from the field of invasion biology into a network of hypotheses. in the following, we will describe this network (section 3.1), provide an overview of the dataset we created from this resource (section 3.2, 3.3), present a qualitative and preliminary quantitative analysis of hypothesis statements (section 3.4) and discuss its intended use (section 3.5). ## hypothesis identification as abstract classification in this section, we report baseline experiments on modeling the automatic identification of hypotheses in the inas dataset. ## conclusion in this work, we proposed and published the inas dataset and conducted initial analyses and experiments on it. our studies revealed interesting in-sights into the availability and distribution of information about the hypotheses in scientific paper titles and abstracts from the field of invasion biology. we believe that there is great potential for a variety of different studies to be performed using this dataset, some of which we plan on conducting in future work. these include further classification experiments like exploring the full hierarchical classification problem, trying to improve classification performance by conducting pretraining on full-texts from the field of invasion biology, or testing one-shot classification leveraging the written hypothesis descriptions. further, our annotation experiment could enable studies on span-level hypothesis detection, e.g. in a weakly-supervised manner or in a one-shot classification setting. finally, we also hypothesize that the introduction of human-engineered knowledge (e.g., in the form of ontologies) into, for example, the classification process can help overcome the problem of a lack of domain-knowledge of current language models.