Divide and Conquer: From Complexity to Simplicity for Lay Summarization (2020)

Abstract: We describe our approach for the 1st Computational Linguistics Lay Summary Shared Task CL-LaySumm20. The task is to produce non-technical summaries of scholarly documents. The summary should be within easy grasp of a layman who may not be well versed with the domain of the research article. We propose a two step divide-and-conquer approach. First, we judiciously select segments of the documents that are not overly pedantic and are likely to be of interest to the laity, and over-extract sentences from each segment using an unsupervised network based method. Next, we perform abstractive summarization on these extractions and systematically merge the abstractions. We run ablation studies to establish that each step in our pipeline is critical for improvement in the quality of lay summary. Our approach leverages state-of-the-art pre-trained deep neural network based models as zero-shot learners to achieve high scores on the task.

## introduction acceptance of science by society is accelerated by sharing scientific knowledge and engaging with the public at large. scientifically backed information, when suitably summarized and conveyed to the common man, spurs empowerment to combat the spread of misinformation. lay summary of a scientific scholarly text, targeted for the general public, captures the broad scientific idea and its potential impact with minimal technical jargon. funding agencies, scientists within and outside the field, * these authors have equal contribution to this work. and science journalists also benefit from lay summaries @xcite . cl-laysumm20 shared task aims to develop nlp methods to bridge the gap between advances made by the scientific community and nonspecialist audience, by summarizing scholarly scientific articles in language understandable by lay persons. evaluation for the task is done on the basis of recall and f1-scores of rouge-1, -2, and -l metrics @xcite . additionally, selective summaries are evaluated by science journalists and communicators for ease of comprehension as well as for interestingness. @xcite document the results and insights from the shared task. ## background and related work earlier works on summarization of scientific articles aim to automatically produce the summary for researchers from multiple perspectives that complement each other. these cover automatic creation of abstract @xcite , extraction of keywords @xcite , title generation (putra and khodra, 2017), extraction of highlights @xcite , queryfocused summarization @xcite and citation based summarization of articles @xcite . various supervised and unsupervised techniques have been used so far for accomplishing distinctive tasks pertinent to scientific articles (altmami and menai, 2020). recently, @xcite propose to leverage the state-of-the-art bert model @xcite for extractive summarization of lectures. in this approach, k-means clustering is performed on sentence embeddings obtained from bert, and the sentences that are closest to cluster centroids are extracted to create the summary. among non-neural models, a popular approach is to capture relations between sentences or word phrases via a weighted graph. @xcite model the sentences of the document as nodes of a weighted directed graph and compute idf based entailment scores between sentence pairs. they use weighted minimum vertex cover to extract most salient sentences. most recent neural abstractive summarizers are trained on masked language modeling task where random sequences of inputs are masked and the model learns to reproduce the masked portions of text. one such model that has achieved state-of-theart results on abstractive summarization datasets is bart @xcite . bart is an autoencoder which is pretrained to reproduce the original input after it has been corrupted with arbitrary noise. bart uses transformer @xcite based architecture that employs selfattention mechanism to allow contextual encoding of input sequence. ## data the organizers provide training and validation corpora for cl-laysumm20 task, named laysumm2 (215 documents) and batch3 (357 documents). these documents comprise abstracts and full texts of scholarly articles from epilepsy, archaeology, and materials engineering domains. each document in the two corpora is accompanied with a gold-standard lay summary. the test set contains 37 documents (abstracts and full texts). al. (2017a) argue that the abstract, being an author generated summary is most important section in a paper. using corpus of 10k computer science research papers, they empirically compare the overlap between different sections and paper highlights. it is reported that among abstract, conclusion, discussion and introduction sections (acdi), introduction section shows least overlap. the authors attribute low importance of introduction section to its longer length. we empirically test this conjecture for lay summaries. we divide the scientific document in two parts -(i) combined acdi text, and (ii) rest of the document and compute the rouge scores of the two parts foot_0 with respect to the gold standard summary. ## experimental setting in this section we describe the choices we made for implementing the lay summarization framework. all the source code is made publicly available on github foot_1 . ## discussion we present two sample system summaries along with the gold standard summaries in appendix b in tables 8 and 9 . we can observe that manual inspection by authors for few other system summaries indicates that we need to improve pre-processing and sentence tokenization. for example, one of the summaries contains confidence interval values which may not be comprehensible to laity. certain inconsistencies in the input format, confuse our parsing algorithms leading to inaccurate segmentation of sections in a few cases. moreover, we notice that in few scenarios, bart leaves out incomplete sentences towards the end, which degrades the quality of lay summary. an astute post-processing check is desirable to address this problem. dependence on abstractive summarizer for the quality of lay summary is the main caveat of the proposed framework. anticipating constant improvement in the state-of-the-art in nlg, we expect the framework to yield high quality lay summaries. ## conclusion we propose a framework for generating lay summaries of scientific scholarly documents. the framework is based on the core idea of extractiveabstractive pipeline to generate lay summaries. we divide the text into segments and focus on information rich segments to extract important sentences. these extracts are fed to the state-of-the-art abstractive summarizer for further compression which improves readability of the summary. this strategy improves the quality of lay summary, while cutting down on the training data requirement as well as computational resources. the proposed framework is frugal in terms of both types of resources. we show that reusing pre-trained publicly available models can be favoured over devising new training architectures. thereby, reaping advantages of transfer learning for specialized tasks.