EVIDENCEMINER : Textual Evidence Discovery for Life Sciences (2020)

Abstract: Traditional search engines for life sciences (e.g., PubMed) are designed for document retrieval and do not allow direct retrieval of specific statements. Some of these statements may serve as textual evidence that is key to tasks such as hypothesis generation and new finding validation. We present EVIDENCEMINER, a web-based system that lets users query a natural language statement and automatically retrieves textual evidence from a background corpora for life sciences. EVIDENCEMINER is constructed in a completely automated way without any human effort for training data annotation. It is supported by novel data-driven methods for distantly supervised named entity recognition and open information extraction. The entities and patterns are pre-computed and indexed offline to support fast online evidence retrieval. The annotation results are also highlighted in the original document for better visualization. EVIDENCEMINER also includes analytic functionalities such as the most frequent entity and relation summarization. EVIDENCEMINER can help scientists uncover important research issues, leading to more effective research and more in-depth quantitative analysis. The system of EVIDENCEMINER is available at https://evidenceminer.firebaseapp.com/ .

## introduction search engines on scientific literature have been widely used by life scientists for discoveries based on prior knowledge. each day, millions of users query pubmed foot_1 and pubmed central foot_2 (pmc) for their information needs in biomedicine @xcite . however, traditional search engines for life sciences (e.g., pubmed) are designed for document retrieval and do not allow direct retrieval of specific statements @xcite @xcite . with the results from those search engines, scientists still need to read a large number of retrieved documents to find specific statements as textual evidence to validate the input query. this textual evidence is key to tasks such as developing new hypotheses, designing informative experiments, or comparing and validating new findings against previous knowledge. while the last several years have witnessed substantial growth in interests and efforts in evidence mining @xcite @xcite @xcite @xcite , little work has been done for evidence mining system development in the scientific literature. a significant difference between evidence in the scientific literature and evidence in other corpora (e.g., the online debate corpus) is that scientific evidence usually does not have a strong sentiment (i.e., positive, negative or neutral) in the opinion it holds. most scientific evidence sentences are objective statements reflecting how strongly they support a query statement. therefore, if scientists are interested in finding textual evidence for "melanoma is treated with nivolumab", they may expect a ranked list of statements with the top ones like "bicytopenia in primary lung melanoma treated with nivolumab" as the textual evidence that supports the input query. this paper presents evidenceminer, a webbased system for textual evidence discovery for life sciences (figure 1 ). given a query as a natural language statement, evidenceminer automatically retrieves sentence-level textual evidence from a background corpora of biomedical literature. evidenceminer is constructed in a completely automated way without any human effort for training data annotation. it is supported by novel data-driven methods for distantly supervised named entity recognition and open information extraction. evidenceminer relies on external knowledge bases to provide distant supervision for named entity recognition (ner) @xcite @xcite . based on the entity annotation results, it automatically extracts informative meta-patterns (textual patterns containing entity types, e.g., chemical inhibit dis-ease) from sentences in the background corpora. @xcite @xcite . sentences with meta-patterns that better match the query statement is more likely to be textual evidence. the entities and patterns are precomputed and indexed offline to support fast online evidence retrieval. the annotation results are also highlighted in the original document for better visualization. evidenceminer also includes analytic functionalities such as the most frequent entity and relation summarization. the contributions and features of the evidenceminer system are summarized as follows. 1. we build evidenceminer, a web-based system for textual evidence discovery for life sciences. evidenceminer is supported by novel methods for distantly supervised named entity recognition and pattern-based open information extraction. 2. the retrieved evidence sentences can be easily located in the original text. the entity and relation annotation results are also highlighted in the original document for better visualization. 3. analytic functionalities are included such as finding the most frequent entities/relations for given entity/relation types and finding the most frequent entities given a relation type with another entity. ## related work search engines performing sentence-level retrieval have been developed in the biomedical domain. for example, textpresso @xcite highlights the query-related sentences in the retrieved documents. however, the sentence highlighting is only based on query word matching, which does not necessarily find sentences semantically related to the input query. another example is litsense @xcite , which retrieves semantically similar sentences in biomedical literature given a query sentence. it returns best-matching sentences using a combined approach of traditional word matching and neural embedding. however, their neural embeddings are noisy and thus negatively impact the effectiveness in retrieving queryspecific evidence sentences. evidenceminer is more effective compared with litsense for textual evidence retrieval in biomedical literature. similar tools are also developed for other domains, such as claim mining and argument mining tools on twitter or news articles. perspectroscope @xcite allows users to query a natural language claim and extract textual evidence in support or against the claim. claimportal (majithia et al., 2019) is an integrated infrastructure for searching and checking factual claims on twitter. targer @xcite is an argument mining framework for tagging arguments in the free input text and keyword-based retrieval of arguments from the argument-tagged corpus. most of these tools rely on fully supervised methods that require human-annotated training data. it is difficult to directly apply these systems to other domains, such as life sciences since it is non-trivial to retrieve the set of human-annotated articles and the annotations are prone to errors @xcite . result visualization in the original document, and ## evaluation to demonstrate the effectiveness of evidencem-iner in textual evidence retrieval, we compare its performance with the traditional bm25 @xcite and a recent sentence-level search engine, litsense @xcite . the background corpus is the same pubmed subset for all the compared methods. we first ask domain experts to generate 50 query statements based on the relationships between three biomedical entity types (gene, chemical, and disease) in the comparative toxicogenomics database foot_7 . then we ask domain experts to manually label the top-10 retrieved evidence sentences by each method with three grades indicating the confidence of the evidence. we use the average normalized discounted cumulative gain (ndcg) score to evaluate the textual evidence retrieval performance. in ## further development in some cases, a strict query matching may not find sufficiently high-quality answers due to the stringent search requirements or limited available entities that match the search queries. in this case, a smart query processor should automatically kick-in to do an approximate match, such as a graph-based approximate match or an embedding-based semantic match. in other cases, a user may query a set of entities (e.g., genes or diseases) or a timeline. we need to conduct a summary of the major differences among the set of entities or over time by analyzing large text. ## conclusion we build evidenceminer, a web-based system for textual evidence discovery for life sciences. the retrieved evidence sentences can be easily located in the background corpora for better visualization. evidenceminer also includes analytic functionalities such as the most frequent entity and relation summarization. we incorporated another corpus on covid-19 in evidenceminer to help boost the scientific discoveries @xcite . we are further developing evidenceminer to be a more intelligent system that can assist in more efficient and in-depth scientific discoveries.