VBD - NLP at B io L ay S umm Task 1: Explicit and Implicit Key Information Selection for Lay Summarization on Biomedical Long Documents (2023)

Abstract: We describe our systems participated in the BioLaySumm 2023 Task 1, which aims at automatically generating lay summaries of scientific articles in a simplified way so that its content becomes easier to comprehend for non-expert readers. Our approaches are based on selecting key information by both explicit and implicit strategies. For explicit selection strategies, we conduct extractive summarization based on selecting key sentences for training abstractive summarization models. For implicit selection strategies, we utilize a method based on a factorized energy-based model, which is able to extract important information from long documents to generate summaries and achieve promising results. We build our systems using sequence-to-sequence models, which enable us to leverage powerful and biomedical domain pre-trained language models and apply different strategies to generate lay summaries from long documents. We conducted various experiments to carefully investigate the effects of different aspects of this long-document summarization task such as extracting different document lengths and utilizing different pre-trained language models. We achieve the third rank in the shared task (and the second rank excluding the baseline submission of the organizers).

## introduction lay summarization is a crucial task and gaining increasing attention due to its potential to provide accessible and digestible scientific information to the general public @xcite . the task involves summarizing technical and specialized content into a readable format for non-expert readers. this task is particularly relevant in biomedical fields, where research findings have significant implications for public health @xcite . in order to help broaden access to technical texts and progress toward more usable abstractive summarization models in the biomedical domain, the biolaysumm 2023 shared task @xcite has been organized for lay summarization task on biomedical research articles. the challenges of this lay summarization task are in two folds: 1) input texts are full articles containing up to 10k sentences, which require models to capture long dependencies and extract key information fragments to generate summaries; 2) the lay summarization task requires us to generate summaries which not only convey the main meaning of the articles but also non-expert vocabularies for readers. we build our systems based on sequence-tosequence models with different key information selection strategies to solve the lay summarization task on biomedical long documents. our abstractive summarization systems are built using sequence-to-sequence (seq2seq) architectures, which have shown state-of-the-art (sota) performance in recent abstractive summarization models @xcite @xcite . in order to deal with the issues of long documents, we focus on two key information selection strategies. specifically, for the first strategy, we explicitly select key sentences as the input for training abstractive summarization models. for the second strategy, long documents are used as inputs and important information is implicitly extracted based on the factorized energy-based model to generate summaries, in which we utilize a model called fac-torsum @xcite , which has shown to be effective in long document abstractive summarization. furthermore, our systems are initialized by biobart @xcite , led @xcite to take advantage of the biomedical domain pre-trained language model. we evaluate our systems by conducting experiments on different aspects such as the effects of sequence length selection, the pre-trained language models, and applying the sota model @xcite . we obtain the best performance with the implicit selection factorsum models and biobart, sepa-rately trained on the two datasets, i.e., plos and elife of the shared task. for the final results on the test set, we achieve the third rank in average scores. for separate metrics, our systems outperform the top teams in three of seven metrics, i.e., the relevance metrics (rouge (1, 2, and l) @xcite , and bertscore @xcite ), the readability metrics (dale-chall readability score (dcrs) @xcite , and flesch-kincaid grade level (fkgl) @xcite ), and the factuality metric (bartscore @xcite ). ## data for the biolaysumm 2023 task 1, the shared task provided two separate datasets, i.e., plos and elife @xcite , containing biomedical journal articles and corresponding expert-written lay summaries. plos: the public library of science (plos) is an open-access publisher that hosts influential peerreviewed journals across all areas of science and medicine. the journals in question focus specifically on biology, computational biology, genetics, pathogens, and neglected tropical diseases. as the data statistics presented in tables 1 and 2 , this task is challenging when we need to generate summaries from long documents with an average from 6k to 10k words in each document, and even more than 25k words in maximum. this requires models dealing with capturing long-range dependencies to extract important fragment information while avoiding out-of-memory issues. for the data sizes, plos contains 24k samples while elife contains only 4k samples. ## our approaches we present two different strategies that we investigate to build our systems to solve this longdocument lay summarization task. ## conclusion we have presented our systems and participated in the biolaysumm shared task to generate lay summaries for long biomedical articles. we approach the task by focusing on the two key information selection strategies: explicitly extracting key sentences to train abstractive summarization models and implicitly extracting important information by utilizing the factorsum model. the results show that the implicit selection model with factorsum obtains the best performance. we achieve the third rank on the test set and obtain several promising results, which outperformed the top teams on several metrics.