Multilingual Simplification of Medical Texts (2023)

Abstract: Automated text simplification aims to produce simple versions of complex texts. This task is especially useful in the medical domain, where the latest medical findings are typically communicated via complex and technical articles. This creates barriers for laypeople seeking access to up-to-date medical findings, consequently impeding progress on health literacy. Most existing work on medical text simplification has focused on monolingual settings, with the result that such evidence would be available only in just one language (most often, English). This work addresses this limitation via multilingual simplification, i.e., directly simplifying complex texts into simplified texts in multiple languages. We introduce MultiCochrane, the first sentence-aligned multilingual text simplification dataset for the medical domain in four languages: English, Spanish, French, and Farsi. We evaluate fine-tuned and zero-shot models across these languages with extensive human assessments and analyses. Although models can generate viable simplified texts, we identify several outstanding challenges that this dataset might be used to address.

## introduction important findings in medicine are typically presented in technical, jargon-laden language in journal articles or reviews, which is difficult for laypeople to understand. this impedes transparent and fair access to critical medical information and ultimately hinders health literacy, which is "one of the most promising and cost-effective approaches to overcome the non-communicable disease challenge" @xcite . text simplification models which automatically transform complex texts into simpler versions understandable by lay readers @xcite have emerged as a promising means of providing wider access to published medical evidence. recent work on simplification has fine-tuned large pre-trained models @xcite figure 1 : examples of our dataset and system outputs for multilingual medical text simplification. the dataset has simplifications available in four languages (english, spanish, french, farsi), and system outputs were analyzed for factual errors (red), fluency errors (blue), and simplicity (green) among other criteria. @xcite @xcite @xcite , explored reinforcement learning @xcite , and evaluated zero-shot performance via prompting @xcite . however, this work has so far exclusively considered monolingual text simplification. consequently, we do not know how well models perform when the simplified text is not in the same language as the original, complex text. this limits the (potential) availability of simplified information to a few high-resource languages-especially for the medical domain-and leads to equity issues such that individuals who do not speak english will still face a high barrier to information access. 1 work in this direction is impeded by a lack of data. the largest resources in text simplification are not in the medical domain, and parallel corpora for medical simplification only exist in single languages, namely english @xcite @xcite , french @xcite , and german @xcite . this paper advances the state of multilingual medical simplification, in which complex texts are to be directly simplified into target languages. our contributions are as follows. we introduce multicochrane (figure 1 ), the first parallel dataset for medical text simplification across multiple languages: english, spanish, french, and farsi. multicochrane contains aligned sentence pairs sourced from the cochrane library of systematic reviews, foot_1 which is a library of meta-analyses of treatment effectiveness. these review articles include both a technical abstract and a plain-language summary (pls), from which we derive the two subsets of multi-cochrane: (1) mc-clean, 101 technical abstracts with expert-annotated manual alignments first derived in english, then semi-automatically aligned to other languages (partially verified by bilingual speakers). mc-clean contains ∼5k sentence pairs across all 4 languages. (2) mc-noisy, a larger but noisier subset created with an automatic sentence alignment model that was trained on mc-clean. mc-noisy is sourced from around 7.8k medical reviews, with ∼100k sentence pairs across languages. multicochrane enables systematic evaluations of medical text simplification models. here we evaluate a range of large pre-trained language models in both zero-shot and fine-tuned settings on the task of text simplification across four languages. in addition to automatic evaluations, we report human assessments covering simplicity, fluency and factuality @xcite . we also report the correlation between automatic metrics and these human assessments, which we find to be mostly weak. our results show that while pre-trained models are effective at simplification in english, their abilities degrade significantly on other languages, to overcome this limitation, it is not ideal due to variability in translation quality across languages @xcite and other issues such as cost and interpretability as discussed in @xcite . this paper evaluates a simplify-then-translate pipeline. we publicly release multicochrane, model outputs, and all human judgments collected for this work ( https://github.com/sebajoe/ multicochrane ), hoping to motivate future work on multilingual medical text simplification to advance model performance across languages. ## related work the largest resources used to train automatic simplification models are two general domain corpora: the wikipedia-simple wikipedia corpus @xcite @xcite , and the newsela corpus @xcite . this paper focuses on simplification in the medical domain, which is important if we are to bridge the information gap exemplified by low medical literacy levels worldwide @xcite . ## the multicochrane corpus we source multicochrane from publicly available technical abstracts and plain-language summaries of cochrane systematic reviews; these are comparable texts, though not parallel @xcite . these abstracts and pls exist in several languages (detailed in sections 3.1.2 and 3.2), aligned almost one-to-one with their english counterparts. this allows annotations and alignments to be easily adapted to create a multilingual dataset. in total, we use 7,755 abstract-pls pairs from the cochrane library. we first create sentence-aligned mc-clean (section 3.1) of 101 abstract-pls pairs using a mixture of manual alignments for english and semi-automatic alignments with partial verification for other languages. the rest of the abstracts were used for the creation of the large-scale noisy mc-noisy (section 3.2) by automatic alignments and filtering. the total number of sentences in all subsets are shown in ## filtering although the crf model handles unaligned sentences by default, the resulting data remains noisy. we therefore adapt the method described in @xcite , initially used in the context of sentence-splitting, to further filter misalignments in mc-noisy. ultimately, this method was applied to the entire training set, which includes the entirety of mc-noisy as well as the portion of mc-clean not used for testing or validation. this method can be described as follows: for each sentence pairing in the training set, we derive the set of lemmatized tokens in the complex and simple sentence denoted by l c and l s respectively. if the proportion of the overlapping tokens from the simplified sentence exceeds a threshold r, it is included in the filtered dataset: we used @xmath0.5 to strike a balance between the number of potential misalignments in the dataset and the extractiveness of the data. overall, as evident in ## evaluation of simplification models multicochrane enables extensive evaluation of multilingual sentence simplification systems simultaneously across 4 languages. in this section we present the first study of this kind (to our knowledge), assessing zero-shot and fine-tuned models (section 5.1) with both automatic metrics (sec-tion 5.2) and human evaluation (section 5.3) for medical text simplification. ## conclusions this paper introduces the first human-aligned multilingual text simplification dataset, mc-clean, for the medical domain for the languages of english, spanish, french, and farsi. we further expanded this dataset using automatic alignments to create a much more vast but noisier dataset, mc-noisy, that could be used for training language models. we also performed an evaluation of multilingual simplification for various systems, testing some of their zero-shot capabilities as well their performance when fine-tuned on mc-noisy.