Readability Controllable Biomedical Document Summarization (2022)

Abstract: Different from general documents, it is recognised that the ease with which people can understand a biomedical text is eminently varied, owing to the highly technical nature of biomedical documents and the variance of readers’ domain knowledge. However, existing biomedical document summarization systems have paid little attention to readability control, leaving users with summaries that are incompatible with their levels of expertise.In recognition of this urgent demand, we introduce a new task of readability controllable summarization for biomedical documents, which aims to recognise users’ readability demands and generate summaries that better suit their needs: technical summaries for experts and plain language summaries (PLS) for laymen.To establish this task, we construct a corpus consisting of biomedical papers with technical summaries and PLSs written by the authors, and benchmark multiple advanced controllable abstractive and extractive summarization models based on pre-trained language models (PLMs) with prevalent controlling and generation techniques.Moreover, we propose a novel masked language model (MLM) based metric and its variant to effectively evaluate the readability discrepancy between lay and technical summaries.Experimental results from automated and human evaluations show that though current control techniques allow for a certain degree of readability adjustment during generation, the performance of existing controllable summarization methods is far from desirable in this task.

## introduction automatic summarization for biomedical documents @xcite such as clinical literature @xcite , provides an efficient way for readers to acquire desirable biomedical information quickly. unlike general documents, biomedical * corresponding author documents have characteristics of mounting scientific jargon @xcite , and complex language structures @xcite . therefore, readers such as non-experts and professionals would seek textual information on different readability levels, since the variance of their biomedical knowledge affects their ease of understanding biomedical papers. for example, an indomain expert might require accurate and clear technical summaries with medical jargon and professional language, to quickly grasp the main contributions of biomedical papers. in contrast, layperson readers usually require plain language summaries with less technical terms and more context of the research, which are easier to understand. nevertheless, current biomedical summarization systems are only able to offer technical abstracts @xcite @xcite or lay language summaries @xcite , fail to generate compatible summaries for various users according to their levels of expertise without considering the readability as an aspect to be controlled during summary generation @xcite . we argue that it is urgent to develop biomedical summarization approaches that can not only condense biomedical documents into concise summaries but also adjust the readability level of summaries to improve the dissemination of scientific information. our research aims to tackle the problem, and thus propose a novel task of readability controllable biomedical document summarization, which is to automatically recognize users' readability demands and generate summaries that are compatible with their expertise level and needs, as shown in figure 1 . specifically, in a binary readability level controlling setting, it is to produce technical summaries for experts, while plain language summaries (pls) for laypeople. the task is challenging since: 1) it requires the model to accurately recognize different readability demands from limited guiding signals, 2) it requires a suitable selection of content from long biomedical documents for various readers guided by their readability demands, 3) it requires the model to learn not only lexical and syntactic adjustment but also paraphrasing according to users' needs. since professionals pay more attention to clarity and accuracy while non-experts prefer summaries that are easier to understand. to approach this task, we build the first corpus consisting of 28,124 biomedical literature with technical and plain language summaries written by the authors, then conduct a thorough analysis of the collected data including statistics, readability metrics, and textual features. next, we examine several controlling techniques on prevalent pre-trained language models (plms) and evaluate their performance on our dataset. apart from automatic assessment, we also bring in the human evaluation due to the inefficacy of current metrics for readability and text generation. to better characterize readability differences between technical summary and pls, we further propose a novel masked noun phrase-based text complexity metric and its variant based on the masked language model (mlm). it is superior in modelling the semantic structure of biomedical texts compared to traditional metrics and existing mlm-based metrics. overall, our main contributions are summarised as follows: (1) we introduce a novel task of readability controllable biomedical document summarization. (2) we build a corpus foot_0 with 28,124 biomedical papers with their technical and plain language summaries, which will facilitate further exploration in this task. (3) we propose an mlm-based text complexity metric, which surpasses existing readability evaluation metrics on our dataset. (4) we examined controlling techniques including prompts and multi-heads on both extractive and abstractive methods to adjust readability during summarization and found the performance is far from satisfying. to the best of our knowledge, this is the first effort to consider readability as a controllable attribute in scientific document summarization. ## baselines for readability controllable summarization existing text summarization methods can be classified into two main ways, extractive and abstractive approaches. we carry out experiments with both methods in this task. ## conclusion in this work, we introduce a novel controlling summarization task that aims at encapsulating biomedical scientific literature on different readability levels. we draw the full text, abstracts, and plain language summaries from papers published in peerreviewed biomedical journals to build the first corpus for the task. we propose an effective mlm-based readability metric and its variant, which outperform traditional and previous mlm-based readability measures in distinguishing technical summaries against plss. we leverage the longformer-encoder-decoder (led) as our skeleton model and examine prevalent generation controlling techniques on both extractive and abstractive methods. we found that though the abstractive approach combined with multi decoders can lead to higherquality summaries and larger textual differences under binary readability demands compared to extractive methods, all tested models fail to show satisfying readability controlling ability. in the future, we would like to investigate how to introduce a more powerful teaching force such as language models trained with corpora of different readability levels to guide the controlling of summarization models.