Capturing Relations between Scientific Papers: An Abstractive Model for Related Work Section Generation (2021)

Abstract: Given a set of related publications, related work section generation aims to provide researchers with an overview of the specific research area by summarizing these works and introducing them in a logical order. Most of existing related work generation models follow the inflexible extractive style, which directly extract sentences from multiple original papers to form a related work discussion. Hence, in this paper, we propose a Relation-aware Related work Generator (RRG), which generates an abstractive related work from the given multiple scientific papers in the same research area. Concretely, we propose a relation-aware multi-document encoder that relates one document to another according to their content dependency in a relation graph. The relation graph and the document representation are interacted and polished iteratively, complementing each other in the training process. We also contribute two public datasets composed of related work sections and their corresponding papers. Extensive experiments on the two datasets show that the proposed model brings substantial improvements over several strong baselines. We hope that this work will promote advances in related work generation task.

## introduction the related work section generation task aims to automatically generate a summary of the most relevant works in a specific research area, which can help researchers to familiarize themselves with the state of the art in the field. several methods @xcite @xcite have been proposed to study how to obtain the related work section automatically by extractive related work: we find that crispr/cas9 can robustly and specifically reduce the expression of these mi-crornas up to 96% @xcite . we find that mirna knockdown phenotypes caused by crispr/cas9 transient editing can be stably maintained in both in vitro and in vivo models for a long term (up to 30 days) @xcite . although genome editing using the crispr-cas system is highly efficient in human cell lines, crispr-cas genome editing in primary human cells is more challenging @xcite . abstractive related work: recently, @xcite showed that crisper-cas9 targeted mirna-17, mirna-200c and mirna-141, repressed their activity in human colon cancer cell lines hct116 and ht-29. furthermore, in vivo targeting was effective for at least a month @xcite . however, off-target mutagenesis and effects of a single mirna on various gene targets are the limitations to the use of this modern technology specifically in brain disorders like prion diseases @xcite . there are two main challenges in this task: (1) the related work should summarize the contribution of each paper, and (2) explain the relationship between different papers such as parallel, turning, and progressive relation, so as to introduce them in a logical order. while existing summarization models can address the first problem, they do not target at comparing and explaining the relationship between these articles. hence, to tackle the above challenges, we propose a relation-aware related work generator (rrg), which generates an abstractive related work given multiple scientific papers in the same research area. firstly, we encode the multiple input articles in a hierarchical manner, obtaining the overall representation for each document. then, we propose a relationaware multi-document encoder that relates multiple input documents in a relation graph. in the training process, the relation graph and the document representation interact and are refined iteratively, complementing each other. finally, in the decoder part, we utilize the relation graph information to assist the decoding process, where the model learns to decide whether to pay attention to the input documents or the relationship between them. to evaluate our model, we introduce two largescale related work generation datasets, which are composed of related work sections and their corresponding papers. extensive experimental results show that rrg outperforms several strong baselines in terms of rouge metrics and human evaluations on both datasets. in summary, our contributions include: ## related work we discuss the related work on related work generation and multi-document summarization. ## related work generation dataset since there are no public large-scale related work generation datasets, we collect two survey datasets composed of related work sections and their corresponding papers. the first dataset is collected from s2orc @xcite , which consists of papers in multiple domains (physics, math, computer sci- ## problem formulation before presenting our approach for related work generation, we first introduce our problem formulation and used notations. to begin with, for a set of relevant papers in a specific area, where d i denotes a paper, we assume there is a corresponding related work @xmath0, y 2 , • • • , y t ). n is the number of relevant papers, and ## conclusion in this paper, we conceptualized the abstractive related work generation task as a machine learning problem. we proposed a new model that is able to encode multiple input documents hierarchically and model the latent relations across them in a relation graph. we also come up with two public large-scale related work generation datasets. experimental results show that our model produces related works that are both fluent and informative, outperforming competitive systems by a wide margin. in the future, we would like to apply our model to abstract generation and paper generation tasks.