ESRA : Explainable Scientific Research Assistant (2021)

Abstract: We introduce Explainable Scientific Research Assistant (ESRA), a literature discovery platform that augments search results with relevant details and explanations, aiding users in understanding more about their queries and the returned papers beyond existing literature search systems. Enabled by a knowledge graph we extracted from abstracts of 23k papers on the arXiv’s cs.CL category, ESRA provides three main features: explanation (for why a paper is returned to the user), list of facts (that are relevant to the query), and graph visualization (drawing connections between the query and each paper with surrounding related entities). The experimental results with humans involved show that ESRA can accelerate the users’ search process with paper explanations and helps them better explore the landscape of the topics of interest by exploiting the underlying knowledge graph. We provide the ESRA web application at http://esra.cp.eng.chula.ac.th/ .

## introduction existing literature search platforms mostly present metadata of papers as search results, and this requires users to read the entire abstracts to understand the brief contents of the returned papers. the users then need to reflect on the knowledge of the papers themselves so as to decide which keywords they should search next. therefore, it is timeconsuming to gradually expand their understanding of the field using existing platforms. meanwhile, research on analyzing scientific literature has been getting more attention due to the extremely large number of new papers published every day @xcite . also, many of them are freely accessible online and the number is still rising @xcite . these lead to several frameworks that aim for extracting knowledge (i.e., scientific concepts and their relations) from scientific documents and representing them as a knowledge graph (kg) @xcite . however, to the best of our knowledge, most of the existing literature platforms have not yet leveraged such extracted knowledge graphs, but only the graph of metadata and hierarchical topics @xcite . so, they are not aware of relations among scientific entities in the papers (e.g., methods, models, and materials) resulting in an inability to provide insightful knowledge beyond a list of papers and abstracts. in this paper, we develop explainable scientific research assistant (esra) -a literature discovery platform that utilizes a knowledge graph and modern natural language processing (nlp) models to augment user experience. esra has three main features built around our extracted knowledge graph as illustrated partly in figure 1 . first, "the explanation feature" explains how the query and each returned paper are related. second, "the fact list feature" suggests top-related keywords with their relationships to the query supporting exploration of related scientific concepts. third, "the graph visualization feature" provides a subgraph illustrating related knowledge around the query and the returned papers. these features aim to help researchers quickly discover and understand a collection of literature they are looking for. the strengths of the main features are demonstrated through a use case in figure 1 . suppose users want to know about "bert", they initially enter "bert" as the search query. on the top of the result page, there is a fact list displayed along with the graph visualization, showing facts (keywords) related to bert such as "bert is a subtype of pre-trained language model" and "bert is used for transfer learning". users can navigate to pages of the related keywords conveniently by clicking the node names as shown in figure 1 (middle-top). from the middle to the bottom of that page, there is a list of returned papers containing their metadata and explanations. for example, the explanation for the paper of roberta @xcite is "we present a replication study of bert pretraining @xcite that carefully measures the impact of many key hyperparameters and training data size. we find that bert was significantly undertrained, and can match or exceed the performance of every model published after it." users can click the paper title in order to redirect to the specific paper page which consists of all available metadata, knowledge graph visualization, references, and citations of the paper. with these features, users can quickly learn about the search query, check out related papers of their interest, and navigate to relevant concepts more conveniently. ## related work in this section, we present an overview of existing work related to esra along two topics, i.e., scientific knowledge extraction frameworks and scientific literature discovery platforms. ## explainable scientific research assistant (esra) our goal is to create a scientific literature discovery platform that is explainable to users and helps them explore and expand knowledge more conveniently. this leads to the esra system with the following three main features, all of which leverage a knowledge graph we extracted from abstracts of the papers in our system. explanation: the explanation attached to each search result enables users to understand the reasons behind the recommendation of the system, i.e., why the paper is selected. the generated explanations for the same paper are dissimilar given different queries, making the explanations become specific to what the users want to know. fact list: for each query, esra displays related knowledge facts from the knowledge graph as a list for the users to explore. the goal of this feature is to aid users in having a better understanding of their search queries. graph visualization: visualization gives users an understanding of the big picture of the relevant knowledge. in both the search result page and individual paper pages, the web application visualizes a subgraph of knowledge that is related to the search keyword and the papers, respectively. to enable these three features, we implemented two main engines underlying esra as shown in figure 2 , including (1) a knowledge graph construction engine and (2) a web application engine. we will explain them and the overall system development in the next subsections. ## results and evaluation we evaluate esra in two ways. first, empirical evaluation concerns the effectiveness of knowledge graph extraction. second, human evaluation targets the three main features of esra -explanation, fact list, and graph visualization. ## conclusion our literature discovery platform, esra, uses a scientific knowledge graph to enhance user's experience. based on the human evaluation, esra can help users screen through papers faster using the generated explanations and capture important facts about the query and the papers using the fact list and the graph visualization. in the future, we aim to expand the coverage of our knowledge graph by extracting facts from the full documents to enhance the quality of esra results.