APTS umm at B io L ay S umm Task 1: Biomedical Breakdown, Improving Readability by Relevancy Based Selection (2023)

Abstract: In this paper we tackle a lay summarization task which aims to produce lay-summary of biomedical articles. BioLaySumm in the BioNLP Workshop at ACL 2023 (Goldsack et al., 2023), has presented us with this lay summarization task for biomedical articles. Our proposed models provide a three-step abstractive approach for summarizing biomedical articles. Our methodology involves breaking down the original document into distinct sections, generating candidate summaries for each subsection, then finally re-ranking and selecting the top performing paragraph for each section. We run ablation studies to establish that each step in our pipeline is critical for improvement in the quality of lay summary. This model achieved the second-highest rank in terms of readability scores (Luo et al., 2022). Our work distinguishes itself from previous studies by not only considering the content of the paper but also its structure, resulting in more coherent and comprehensible lay summaries. We hope that our model for generating lay summaries of biomedical articles will be a useful resource for individuals across various domains, including academia, industry, and healthcare, who require rapid comprehension of key scientific research.

## introduction the sharing of scientific knowledge and engaging with the public is vital for accelerating the acceptance of science in society. the process of generating lay summaries manually can be timeconsuming and challenging, especially for authors who may be unfamiliar with communicating their findings to non-specialist audiences. when scientifically backed information is summarized and conveyed in layman's terms, it empowers people to combat the spread of misinformation. this is our project aims to bridge the gap between the increasing availability of health information and the difficulty the public has understanding it. working towards this direction we took part in the biolay-summ 2023. we participated in task 1 in which given an article's abstract and its important sections as input, the goal is to train a model to generate the lay summary. this shared task also emphasized the importance of paragraph readability in the evaluation of lay summaries @xcite . the training and evaluation of our model were carried out on two distinct datasets, namely plos and elife @xcite . these datasets consist of biomedical research articles, along with their corresponding technical abstracts and expert-written lay summaries. the metrics used for evaluation are divided in 3 parts, 1) relevance : this included rouge (1, 2, and l) @xcite ) -recall based metrics and bertscore @xcite , 2) readability : flesch-kincaid grade level (fkgl) @xcite and @xcite , @xcite 3) factuality : bartscore @xcite . in case of relevance and factuality higher scores mean good result, but for readability lower scores are preferred. the checkpoints for bertscore and bartscore were provided by the organization. considering the potential significance of each section, we are using the structure of the article based on the relevancy, then incorporating a re-ranking based approach, ( some of which is discussed in the paper simcls @xcite ) on the sections chosen on the basis of their relevancy to the target summary. then joining the section-wise summaries generated in a pre-determined manner to generate the final summary. the pre-determined manner is finalized after various experiments and the method we arrived at effectively captures the essential information. our system outperforms existing methods in terms of readability, achieving the second-highest score on this metric. also, our pipeline performs well on the commonly used relevancy metrics (rouge scores) for summarization. we attribute this success to leveraging the document structure and careful selection of the most informative parts in each section. ## related works in this section we will discuss ideas related to summarization and structure based summarization. ## methodology our pipeline has a three-step abstractive approach for summarizing biomedical articles. our methodology entails breaking down the original document into distinct sections, selecting relevant sections and then generating candidate summaries for each section, and then re-ranking and selecting each section's k-top-performing candidate-summaries. last two steps are covered with the help of the model simcls with the adjustment of the seq2seq model used to generate candidate summaries within it. we subsequently join the top summaries of each section with the purpose of finding the best combination. overall pipeline is shown in figure 1 ). ## experimental settings fine-tuning of bart ran for 2 epochs per section, the maximum token length was set to 1024, and a batch size of 2 was used. runtime per epoch of each section in elife was 6 minutes and for plos 35 minutes. then for candidate generation batch size was set to 8 and, per section it took 13 hours 12 minutes for plos and 2 hours 36 minutes for elife. finally for re-ranking, the batch size was 8 and per section it took 44 minutes for plos and 13 minutes for elife. for re-ranking portion of inference on the validation and test sets, each sample took 0.11 sec. ## results and discussion the proposed approach for summarizing biomedical articles in the context of lay summaries has yielded promising results. here, we present the outcomes of our experiments the baseline score presented in ## conclusion in conclusion, we proposed a re-ranker based model which leverages the document structure and specifically filters out the most important sections. our model achieved the second-highest rank in terms of readability scores in the biolaysumm shared task of the bionlp workshop at acl 2023. we hope that the development of these models will continue to play a critical role in advancing healthcare and will be a valuable resource for individuals across various domains.