Making Science Simple: Corpora for the Lay Summarisation of Scientific Literature (2022)

Abstract: Lay summarisation aims to jointly summarise and simplify a given text, thus making its content more comprehensible to non-experts.Automatic approaches for lay summarisation can provide significant value in broadening access to scientific literature, enabling a greater degree of both interdisciplinary knowledge sharing and public understanding when it comes to research findings. However, current corpora for this task are limited in their size and scope, hindering the development of broadly applicable data-driven approaches. Aiming to rectify these issues, we present two novel lay summarisation datasets, PLOS (large-scale) and eLife (medium-scale), each of which contains biomedical journal articles alongside expert-written lay summaries.We provide a thorough characterisation of our lay summaries, highlighting differing levels of readability and abstractivenessbetween datasets that can be leveraged to support the needs of different applications.Finally, we benchmark our datasets using mainstream summarisation approaches and perform a manual evaluation with domain experts, demonstrating their utility and casting light on the key challenges of this task.

## introduction scientific publications contain information that is essential for the preservation and progression of our understanding across all scientific disciplines. typically being highly technical in nature, such articles tend to assume a degree of background knowledge and make use of domain-specific language, making them difficult to comprehend for one lacking the required expertise (i.e., a lay person). these factors often limit the impact of research to only its direct community @xcite and, more 1 * corresponding author. ## related work past attempts to automatically summarise scientific content in layman's terms have been scarce, with the most prominent example being the laysumm subtask of the cl-scisumm 2020 shared task series (chandrasekaran et al., 2020) which attracted a total of 8 submissions. alongside the task, a training corpus of 572 articles and author-generated lay summaries from a multi-disciplinary collection of elsevier-published scientific journals was provided, with submissions being evaluated on a blind test set of 37 articles. it was noted by the task organisers that the data provided was insufficient for training a model to produce a realistic lay summary. @xcite also make use of a single publication source to retrieve lay summaries: the cochrane database of systematic reviews (cdsr). their dataset contains the abstracts of 6,695 systematic reviews paired with their respective plain-language summaries, covering various healthcare domains. although larger than other available datasets for lay summarisation, cdsr is constrained in that it only uses the abstracts of systematic reviews as source documents, and thus models trained using cdsr will be unlikely to generalise well to inputs that are longer than an abstract or the abstracts of other types of publication. alternatively, zaman et al. (2020) introduce a dataset derived from the 'eureka-alert' science news website for the combined tasks of simplification and summarisation. summaries consist of news articles (average @xmath0. however, the extensive size of reference summaries is likely to present additional challenges in model training and their news-based format limits their applicability (e.g., in automating lay summarisation for journals). compared to previous resources, our datasets contain articles and lay summaries of a format that we consider to be more broadly applicable to wider literature. additionally, plos is significantly larger than those currently available (over 4× larger than cdsr) and elife contains sum- ## our datasets we introduce two datasets from different biomedical journals (plos and elife), each containing full scientific articles paired with manually-created lay summaries. for each data source, articles were retrieved in xml format and parsed using python to retrieve the lay summary, abstract, and article text. 1 in line with previous datasets for scientific summarisation @xcite , the article text is separated into sections, and the heading of each section is also retrieved. sentences are segmented using the pysbd rule-based parser @xcite , which we empirically found to outperform neural alternatives. we separate our datasets into training, validation, and testing splits at a ratio of 90%/5%/5%. statistics describing the contents of our datasets and that of past lay summarisation datasets are given in plos the public library of science (plos) is an open-access publisher that hosts influential peerreviewed journals across all areas of science and medicine. several of these journals require authors to submit an author summary alongside their work, defined as a 150-200 word non-technical summary aimed at making the findings of a paper accessible to a wider audience, including non-scientists. 2 the journals in question focus specifically on biology, computational biology, genetics, pathogens, and neglected tropical diseases. ## dataset analysis we carry out several analyses comparing the lay summaries of our datasets to the respective technical abstracts. through these analyses, we seek to highlight and quantify the key differences between these two different types of summary, as well as those present between the lay summaries of our two datasets. specifically, we focus on readability ( §4.1), rhetorical structure ( §4.2), vocabulary sharing ( §4.3), and abstractiveness ( §4.4). ## experiments and results to help facilitate future work, we benchmark our datasets using popular heuristics-based, unsupervised, and supervised summarisation approaches ( §5.1). additionally, we provide further insight into these results via a detailed discussion ( §5.2) and an expert-based manual evaluation ( §5.3). 7 better illustrated by ## conclusion in this work, we have introduced plos and elife, two new datasets for the lay summarisation of biomedical research articles. compared to currently available resources, these datasets possess source article and summary formats that are more broadly applicable to wider literature, with plos also being larger by a significant margin. a thorough analysis of our lay summaries highlights key differences between datasets, enabling them to cater to the needs of different audiences and applications. specifically, in addition to being approximately twice as long as those of plos, we find elife summaries to be both more readable and abstractive, thus better suited to a less technical au-dience. to facilitate future research, we benchmark our datasets with popular summarisation models using automatic metrics and conduct an expert-based human evaluation, providing further insight into the intricacies of model performance on our datasets and highlighting key challenges for the task of lay summarisation.