Comparative Graph-based Summarization of Scientific Papers Guided by Comparative Citations (2022)

Abstract: With the rapid growth of scientific papers, understanding the changes and trends in a research area is rather time-consuming. The first challenge is to find related and comparable articles for the research. Comparative citations compare co-cited papers in a citation sentence and can serve as good guidance for researchers to track a research area. We thus go through comparative citations to find comparable objects and build a comparative scientific summarization corpus (CSSC). And then, we propose the comparative graph-based summarization (CGSUM) method to create comparative summaries using citations as guidance. The comparative graph is constructed using sentences as nodes and three different relationships of sentences as edges. The relationship that sentences occur in the same paper is used to calculate the salience of sentences, the relationship that sentences occur in two different papers is used to calculate the difference between sentences, and the relationship that sentences are related to citations is used to calculate the commonality of sentences. Experiments show that CGSUM outperforms comparative baselines on CSSC and performs well on DUC2006 and DUC2007.

## introduction today, the transient and rapidly evolving research areas and the numerous published research articles require researchers to orient themselves and discover the changes of the research area @xcite . in order to reduce the burden of researchers, a solution is to find and compare related articles in the research area, and automatically create comparative summaries showing commonalities and differences of the articles where differences mean changes. the first problem is how to find related and comparable articles, and the second problem is how to create comparative summaries. figure 1 : an example from our dataset. the comparative citation appears in the citing paper @xcite and cites two cited papers @xcite . the comparative summary is generated by summarizing the commonality (italicized) and difference (underlined) between three papers regarding to the comparable topic mentioned in the citation. fortunately, comparative citations can serve as good guidance in finding related and comparable articles and common topics. according to previous works @xcite , the function of comparative citations is to include shared topics between papers in the same field and reflects the comparative intent, i.e., the author intends to compare his own work with cited works. citation function has been widely investigated. @xcite analyzed citation functions based on empirical works, and classified functions as contrast, neutral, weakness, etc. among all citation functions, the comparative citation is most suitable for comparative summarization as it contains most comparative information. given a set of comparable articles guided by comparative citations, we aim to summarize commonalities and differences between the articles and related to the comparable topics mentioned in citations. as the figure 1 shows, the comparative citation in the upper part is captured by "unlike", where the citing and the cited papers share the comparable topic. also, the bottom part shows a comparative summary based on mentioned comparative citation. the summary accommodates the commonality and difference between the citing and the cited papers. our task is different from traditional survey generation and related work generation @xcite @xcite @xcite in that 1) our task utilizes the citing and the cited papers guided by comparative citations, and 2) related work and surveys focus on shared information while comparative summaries capture commonalities and differences. we build a comparative scientific summarization corpus (cssc) based on comparative citations. three annotators are asked to annotate and collect comparative citations in 32 papers in the ai area using the citation function annotation scheme. we get 40 comparative citations with the corresponding citing and cited papers. for each comparative citation, annotators read through papers to generate a draft comparative summary for the comparative topic mentioned in the citation. after that, five postgraduates students specializing in works on selected 32 papers read and revise the draft comparison summary to create the ground truth. since our dataset is small-scaled, we propose a simple yet effective unsupervised comparative graph-based scientific summarization method (cg-sum). a comparative graph is built to represent the citation texts and papers. each paper or citation text corresponds to a subgraph, where nodes represent sentences and weights of edges denote similarities between sentences. the salience of a sentence is computed by considering the position of the sentence within the paper. the commonality of a sentence is computed on its subgraph and the citation texts subgraph. the difference of a sentence is captured by adding negative edges between nodes from different paper subgraphs. finally, salience, commonality and difference are linearly combined to rank and select sentences. experiments show that cgsum outperforms baselines on cssc and also performs well on duc2006 and duc2007. our contributions are summarized as follows: ## related work citations throughout scientific papers help understand the frontiers and trends in diverse research fields. @xcite analyzed citation functions based on empirical works, which is similar to @xcite . whereas dong and schäfer (2011 generic scientific summarization uses extractive @xcite @xcite , abstractive @xcite @xcite and other @xcite @xcite methods to summarize a document. citation generation has also been studied @xcite . early studies were based on keywords @xcite . xing et al. (2020) considered the abstract of cited papers to generate citations. citation generation concerns semantics, while comparative summarization concerns citation function. the citation text is often too short to describe in detail. related work generation and survey generation generates from multiple documents. he et al. (2016) captured hot topics in fields. chen and zhuge (2019); abura'ed et al. (2020); abura'ed and saggion (2021) took citations into account to mine information. what's more, wang et al. (2020); yuan et al. (2021) generated reviews that cover more aspects. chen et al. (2021) took abstractive method. related work generation places emphasis on shared content in cited papers and summarizes the common information. however, it is different from our task of comparative summarization, which is guided by comparative citations and summarizes commonalities and differences. ## task definition given the comparative citation (cit), the citing paper (cp) where the cit appears, and a set of reference papers (rps) that the cit cites, the task aims to create the comparative summary containing commonalities and differences between cp and rps with regard to the comparable topic mentioned in the cit. taking the case in figure 1 , cit refers to the comparative citation, cp refers to the @xcite and rps refers to @xcite . the comparable topic mentioned in the cit is language representation models. since there are no existing datasets and methods for the task, we build a comparative citationguided dataset and propose a comparative graphbased method. to create the dataset, we annotate the function of citations of 32 papers in the ai area, and manually create comparative summaries for 40 annotated comparative citations. then we create the comparative summary, the proposed summarization method leverages different relationships between sentences to construct a comparative graph and extract sentences from papers. ## dataset construction this section mainly contains citation annotating as well as data processing. figure 2 depicts an overview of the data construction process. ## comparative graph-based scientific summarization we propose the comparative graph-based summarization method (cgsum for short). the overview of the method is shown in figure 3 . the core idea of the method is to select sentences by calculating the salience of the sentences and estimating the degrees to which the sentences reflect the commonalities and differences between papers. ## conclusion this paper proposes the novel task of comparative citation-guided summarization of scientific papers, which aims to summarize commonalities and differences between the articles and related to the comparable topic mentioned in comparative citations. the cssc dataset for the task is constructed, which contains 40 groups of comparable scientific papers and corresponding reference summaries by annotating and collecting comparative citations. the unsupervised comparative graph-based summarization cg-sum method is proposed to generate comparative summaries. it utilizes three different relationships of sentences to build a comparative graph and calculates the scores of salience, commonality and difference without large-scaled data. experiments on cssc show that cgsum outperforms baselines. experiments on duc2006 and duc2007 demonstrate that cgsum can be generalized to multi-document summarization tasks. in the future, we would like to study more types of relationships between documents and research the comparative scientific summarization cross the fields.