Paragraph-level Simplification of Medical Texts (2021)

Abstract: We consider the problem of learning to simplify medical texts. This is important because most reliable, up-to-date information in biomedicine is dense with jargon and thus practically inaccessible to the lay audience. Furthermore, manual simplification does not scale to the rapidly growing body of biomedical literature, motivating the need for automated approaches. Unfortunately, there are no large-scale resources available for this task. In this work we introduce a new corpus of parallel texts in English comprising technical and lay summaries of all published evidence pertaining to different clinical topics. We then propose a new metric based on likelihood scores from a masked language model pretrained on scientific texts. We show that this automated measure better differentiates between technical and lay summaries than existing heuristics. We introduce and evaluate baseline encoder-decoder Transformer models for simplification and propose a novel augmentation to these in which we explicitly penalize the decoder for producing “jargon” terms; we find that this yields improvements over baselines in terms of readability.

## introduction the need for accessible medical information has never been greater. a pew research survey of american's online health habits in 2013 revealed that "one in three american adults have gone online to figure out a medical condition" @xcite . given the rise of medical misinformation on the internet @xcite , accessibility has become an increasingly urgent issue @xcite . however, sources that provide accurate and up-to-date information, including scientific papers and systematic reviews @xcite , are often effectively inaccessible to most readers because they are highly technical and laden with terminology @xcite . technical abstract: analysis showed a higher rate of weight gain in the high-volume feeds group: mean difference 6.20 g/kg/d (95% confidence interval 2.71 to 9.69). there was no increase in the risk of feed intolerance or necrotising enterocolitis with high-volume feeds, but 95% confidence intervals around these estimates were wide. plain-language summary: very low birth weight infants who receive more milk than standard volumes gain weight more quickly during their hospital stay. we found no evidence suggesting that giving infants high volumes of milk causes feeding or gut problems, but this finding is not certain. one potential solution to this problem is text simplification, i.e., editing documents such that they are accessible to a wider audience, while preserving the key information that they contain. although manual simplification is too expensive to feasibly apply at scale, automatic text simplification @xcite provides a potential means of rendering a large volume of specialist knowledge more accessible. large-scale data-driven simplification systems have mostly been trained on wikipedia @xcite @xcite and news @xcite , and focus on sentence simplification @xcite @xcite @xcite @xcite ; on the other hand, medical text simplification is resource poor. recent work has involved constructing sentencealigned data automatically using monolingual text alignment methods @xcite , but this process is noisy and constrains the task to sentence-level simplification. in this work we explore new data and modern conditional text generation models @xcite to simplify medical documents. we introduce a dataset of paired (technical, simplified) texts derived from the cochrane database of systematic reviews, which is comprised of evidence syntheses on a wide range of clinical topics. critically, each review includes a plain-language summary (pls) written by the authors. pls are written directly from the full reviews with their own structure and guidelines; they are not simplified versions of the corresponding technical abstracts of the reviews, nor are they summaries of the abstracts. however, we observe that portions of the pls can be considered simplifications of analogous sections in the abstracts, that is, they contain roughly the same content but involve simplification operations such as paraphrasing, word/sentence deletion, and summarization. we heuristically derive 4459 such pairs of sections (or paragraphs) of technicalplain english bitexts. we provide an excerpt of the dataset we have constructed in this data allows us to explore characteristics of simplified versions of technical medical texts. we show that the differences in traditional readability metrics, such as flesch-kincaid @xcite and automated readability index (senter and smith, 1967), are small. instead, the differences are better captured using large-scale pretrained masked language models, and this reveals that there is more to the language difference than the shallow cues such as sentence and word lengths that traditional readability metrics focus on. we present baseline methods for automatic text simplification over this data and perform analyses that highlight the challenges of this important simplification task. we find that when naively finetuned for the task, existing encoder-decoder models such as bart @xcite tend to prefer deletion over paraphrasing or explaining, and are prone to generating technical words. we propose a new approach to try and mitigate the latter issue by imposing a variant of unlikelihood loss @xcite that explicitly penalizes the decoder for production of 'technical' tokens. we show that this yields improvements in terms of readability with only a minor tradeoff with content quality. in sum, this work takes a step towards paragraphlevel simplification of medical texts by: (1) introducing a sizable new dataset, (2) proposing and validating a new masked language model (mlm)based metric for scoring the technicality of texts, (3) analyzing and understanding the style of plain language in this important domain, and (4) presenting baselines that exploit a variant of unlike-lihood training to explicitly penalize models for producing jargon. we release our code and data at https://github.com/ashologn/paragraphlevel-simplification-of-medical-texts . ## related work recent efforts on data-driven text simplification methods have tended to rely on two resources: the wikipedia-simple wikipedia aligned corpus @xcite @xcite and the newsela simplification corpus @xcite . yet, there is an urgent need to simplify medical texts due to health literacy levels @xcite . however, due to a lack of resources with which to train model-based simplification systems in this domain, past work has tended to focus on lexical simplification @xcite @xcite . @xcite and @xcite introduced sentence-aligned corpora at the scale of thousands of sentence pairs. in contrast to our corpus, these datasets were automatically derived using paraphrase mining or monolingual alignment processes. furthermore, as these are exclusively sentence corpora, they limit the set of potential approaches to just those that operate over sentences. @xcite created a simplification corpus for medical texts in french, in which a small subset of the text pairs are manually sentence-aligned, resulting in 663 sentence pairs, 112 of which are also from cochrane. with respect to modeling, recent work has focused on sentence simplification, treating it as a monolingual machine translation task @xcite @xcite using encoder-decoder models @xcite @xcite . in the medical domain, existing systems tend to adopt lexical and syntactic simplification @xcite @xcite . research on document simplification has been sparse; to the best of our knowledge, the few prior works on this in english have focused on analysis @xcite , sentence deletion (woodsend and lapata, 2011; @xcite , and localized explanation generation @xcite . this work proposes and evaluates an encoder-decoder model for paragraph-level simplification. ## technical abstracts vs. plain-language summaries we compiled a dataset of technical abstracts of biomedical systematic reviews and corresponding pls from the cochrane database of systematic reviews, which comprises thousands of evidence synopses (where authors provide an overview of all published evidence relevant to a particular clinical question or topic). the pls are written by review authors; cochrane's pls standards @xcite recommend that "the pls should be written in plain english which can be understood by most readers without a university education". pls are not parallel with every sentence in the abstract; on the contrary, they are structured heterogeneously @xcite . ## results in this section we comment on the generated texts' readability, quality of summarization and simplification, stylistic fidelity with the pls, and overall coherence and simplicity based on human examination. in the results tables, we indicate whether lower or higher scores for the metrics reported are better with ↓ and ↑ symbols, respectively. ## conclusions in this work we considered the important task of medical text simplification. we derived a new resource for this task made up of technical abstracts summarizing medical evidence paired with plain language versions of the same; we have made this data publicly available to facilitate further research. 3 we proposed a new masked language model (mlm)-based measure of the technicality of text, which quantifies technicality by calculating the likelihood of tokens in the input text with respect to a transformer-based mlm trained on a technical corpus. we demonstrated that this metric better discriminated technical abstracts from pls than more traditional notions of readability. we proposed models for automated simplification based on bart @xcite , extending the training objective by incorporating an explicit penalty for production of 'jargon' terms. we found that this method can improve model outputs (i.e., can increase simplicity and the abstractiveness of summaries) according to the metrics considered.