NCUEE - NLP at B io L ay S umm Task 2: Readability-Controlled Summarization of Biomedical Articles Using the PRIMERA Models (2023)

Abstract: This study describes the model design of the NCUEE-NLP system for BioLaySumm Task 2 at the BioNLP 2023 workshop. We separately fine-tune pretrained PRIMERA models to independently generate technical abstracts and lay summaries of biomedical articles. A total of seven evaluation metrics across three criteria were used to compare system performance. Our best submission was ranked first for relevance, second for readability, and fourth for factuality, tying first for overall performance.

## introduction automatic text summarization is a nlp task that aims to generate concise, relevant, and informative summaries of a source document. the goal is to provide readers with a quick and easy way to understand the essential content of a document without having to read the entire text. different from extractive summarization techniques that select important sentences from the original texts to create a summary, abstractive summarization methods learn a semantic representation of the source content to generate a summary that conveys the meaning of original texts in a more concise way. biomedical publications usually contain a wide range of health-related topics and may be of interest to researchers, medical experts, journalists and even members of the general public. however, such articles tend to include considerable amounts of technical jargon which can present a significant comprehension barrier to non-expert readers. abstractive summarization models have the potential to help broaden access to biomedical articles by highlighting key information in less technical language. the biolaysumm shared task @xcite at the bionlp 2023 workshop seeks to generate more readable summaries of biomedical articles (i.e., a "lay summary"). we participated in the task 2 focused on readability-controlled summarization, where given the main text of a biomedical article as input, the goal is to develop a single model to generate both a technical abstract and a lay summary. this paper describes the ncuee-nlp (national central university, dept. of electrical engineering, natural language processing lab) system for the biolaysumm task 2 @xcite . our solution explores the use of a pretrained primera model and fine-tuning on the downstream summarization task for technical abstract and lay summary generation. each participating team was allowed to submit a maximum of three runs. the three evaluation criteria contained seven metrics used for performance comparisons. our best performing submission ranked first for relevance, second for readability, and fourth for factuality, and tied first for overall performance. the rest of this paper is organized as follows. section 2 describes the ncuee-nlp system for biolaysumm task 2. section 3 presents the results and performance comparisons. conclusions are finally drawn in section 4. ## the ncuee-nlp system figure 1 shows our ncuee-nlp system architecture for the biolaysumm task 2. we use the primera model @xcite , which is a pre-trained model focused on multi-document summarization, as our main architecture to independently generate a technical abstract and a lay summary for this task. primera @xcite was designed to reduce the needs for dataset-specific architectures and large fine-tunning labeled data. following the gap sentence generation (gsg) objective used in pegasus @xcite , a new pretraining strategy called entity pyramid was proposed to train the model to identify and aggregate salient information based on entity frequency across related document clusters. different from the importance heuristic in pegasus, the entity pyramid strategy selects masked sentences that are representative of more documents in the cluster rather than exact matching between documents. primera uses the longformer-encoder-decoder (led) @xcite as the model initialization. the length limit of input was set at 4096 with a sliding window size of 512 for local attention. the output length limit was configured as 1024. extensive experiments on 6 datasets from 3 different domains showed that primera outperformed other state-of-the-art models with zero-shot, few-shot, and fully-supervised settings. this motivated us to explore how the primera model can be used to effectively address this task challenge. biomedical articles are comprised of a sequence of sections such as introduction, materials, method, results, and conclusions, wherein each section is regarded as a document to be delimited with a @xmath0. the acknowledgments and references sections were excluded. tokens that exceed the input length limit were automatically discarded. it's identical to the original primera model @xcite , using a local and global attention mechanism in the encoder self-attention side while using full attention on the decoder and cross-attention. we used the training set to separately fine-tune the pretrained primera models to generate the technical abstract and lay summary. during the evaluation phase, we independently used the primera model to generate the abstract and summary as our model outputs and group them for final system result submission. ## conclusions this study describes the ncuee-nlp system for the biolaysumm task 2, including system design, implementation and evaluation. each section in a biomedical article is regarded as a document and kept its original sequence in paper organization for multi-document summarization based on the primera model. we fine-tuned the pretrained language model using the training set, achieving good performance on the test set for relevance and readability, while improvements are still needed in terms of factuality. finally, our submission tied for first place for overall performance.