Dimsum @ L ay S umm 20 (2020)

Abstract: Lay summarization aims to generate lay summaries of scientific papers automatically. It is an essential task that can increase the relevance of science for all of society. In this paper, we build a lay summary generation system based on BART model. We leverage sentence labels as extra supervision signals to improve the performance of lay summarization. In the CL-LaySumm 2020 shared task, our model achieves 46.00 Rouge1-F1 score.

## introduction nowadays, researchers have been increasingly tasked by funders and publishers to outline their research for the public by writing a lay summary. therefore, it is essential to automatically generate lay summaries to reduce the workload for researchers as well as build a bridge between the public and science. previous studies have investigated scientific article summarization especially for papers @xcite @xcite . however, less work has been done to generate lay summaries. recently, the first workshop on scholarly document processing (chandrasekaran, 2020), lay summary task 1 (laysumm 2020) first proposed the task of lay summary generation. the task aims to generate summaries that are representative of the content, comprehensible and interesting to a lay audience. after checking the dataset that the task provides, we observe that lots of the sentences in lay summaries have corresponding sentences in original papers. inspiring by this observation, we think that making binary sentence labels for extractive summarization and utilize them as extra supervision signals can help model generate better summaries. therefore, we conduct bart (lewis 1 https://ornlcda.github.io/sdproc/index.html @xcite encoder to make sentence representations and train extractive summarization together with abstractive summarization. experimental results show that leveraging sentence labels can improve the lay summary generation performance. in the laysumm 2020 competition, our model achieves 46.00% rouge1-f1 score. the code will be released on github foot_0 . ## related work text summarization text summarization aims to produce a condensed representation of input text that captures the core meaning of the original text. recently, neural network-based approaches have reached remarkable performance for news articles summarization @xcite @xcite . comparing with news articles, scientific papers are typically longer and contain more complex concepts and technical terms. scientific paper summarization existing approaches for scientific paper summarization include extractive models that perform sentence selection @xcite @xcite and hybrid models that select the salient text first and then summarize it @xcite . besides, @xcite built the first model for abstractive summarization of single, longer-form documents (e.g., research papers). in order to train neural models for this task, several datasets have been introduced. the arxiv and pubmed datasets @xcite were created using open access articles from the corresponding popular repositories. @xcite developed and released the first large-scale manuallyannotated corpus for scientific papers (on computational linguistics). large pre-trained language model large pretrained language models, such as bert @xcite , unilm @xcite and bart @xcite have shown great performance on a variety of downstream tasks including summarization. for example, bart achieved state-ofthe-art performance on cnn/dm @xcite news summarization dataset. ## datasets we use two datasets for this work, which are the dataset of cl-laysumm 2020 and scisummnet @xcite . in this section, we introduce the details of them and the pre-processing method we used. ## experiments during the training phase, we randomly select 90% of the cl-laysumm 2020 dataset for training and 10% for validation. if a data sample doesn't contain an abstract or introduction, we don't include it in training or validation. to find the optimal architecture for this task within the models we have, we set up seven different experiments. ## result analysis the results are shown in different inputs to the model. the experiment results of bart (abs), bart (abs+intro), and bart (abs+intro+con) show by adding the introduction and conclusion to the input, the models' performance improves consistently. however, comparing with the results from bart (abs+intro) and bart (abs+intro all ), using the whole introduction 3 https://github.com/huggingface/transformers rather than the first paragraph of the introduction decreases the performance on rouge1 score. we think it is because the cl-laysumm 2020 task requires to make a relatively short summary, less than 150 words. if the input is too long, it makes the model harder to summarize because longer input contains more noisy data. since the cl-laysumm 2020 dataset is also small, the model doesn't have enough samples to learn the task. ## conclusion in this paper, we showcased how different inputs, data augmentation, training strategy, and sentence labels influence the lay summarization task. we introduce a new method to utilize sentence labels as another supervision signal while training bart based model. experimental results show our models can generate better summaries evaluated by the rouge1-f1 score.