F rench Biomedical Text Simplification: When Small and Precise Helps (2020)

Abstract: We present experiments on biomedical text simplification in French. We use two kinds of corpora – parallel sentences extracted from existing health comparable corpora in French and WikiLarge corpus translated from English to French – and a lexicon that associates medical terms with paraphrases. Then, we train neural models on these parallel corpora using different ratios of general and specialized sentences. We evaluate the results with BLEU, SARI and Kandel scores. The results point out that little specialized data helps significantly the simplification.

## introduction the goal of text simplification is to make complex texts accessible to a given target audience (children, foreigners, diseased people...) or to make complex texts to be more easily processed by nlp applications. there is little existing work on biomedical text simplification in english @xcite , and on text simplification for the general language in french @xcite @xcite . research on text simplification is usually performed on open domain english. currently used methods rely on deep learning approaches and require large parallel monolingual corpora in which one complex sentence is paired with one or more simplified versions @xcite . two english datasets are commonly used: (1) newsela @xcite , a corpus with news articles that are manually re-written according to four levels of simplification, and (2) wikilarge @xcite , issued from a compilation of three previously released simplification corpora all extracted from wikipedia @xcite @xcite . the availability of corpora and resources plays indeed a very important role and condition the feasibility of the nlp research. we build and test models for biomedical text simplification in french. we first describe the data used (section 2) and various configurations of the experiments (section 3). we then indicate the evaluation principles (section 4), and present and discuss the results (section 5). we also provide the wikilarge fr corpus and a set of native parallel sentences in french, from general and biomedical languages. we use two sets of parallel corpora dedicated to the simplification. one set is obtained from the freely available simplification corpus clear in french @xcite . this is a comparable corpus, which contains three types of texts (medical literature reviews, drug information, and medical articles from wikipedia and vikidia), and from which parallel sentences were extracted. this parallel corpus contains 4,596 sentence pairs, which is not sufficient when using the current state-of-the-art methods, that rely on deep learning. another corpus is obtained thanks to the automatic translation of wikilarge in french. the translation has been done using opennmt-py @xcite with the default parameters, and the en-fr model provided. this wikilarge fr parallel corpus contains almost 300,000 sentence pairs. ## linguistic data we also use a lexicon that proposes layman paraphrases for technical medical terms, like {hypotension; baisse de la tension artérielle} ({hypotension; decrease in arterial pressure} ). this lexicon has been built using medical terminologies @xcite and corpora (clear corpus and various discussion fora) in french. the lexicon currently contains 7,580 paraphrases for 4,516 medical terms. ## experimental protocol the purpose of the experimental protocol is to focus on two aspects: we refer to the first set of experiments as npt (no phrase table), the second set of experiments as pts (phrase table used in the simplification phase) and the third set of experiments as ptt (phrase table added to the training set). across each series of experiments, the validation and test sets are the same, while the training sets differ in the ratios and whether or how the lexicon is used. we use opennmt-py for the sentence simplification with the following configuration: two bidirectional lstm layers of 500 units for the encoder and the decoder, adam optimizer, learning rate 0.001, dropout probability 0.3, attention dropout probability 0.2. each individual training took about five hours on a geforce rtx 2070 gpu. during the simplification phase, we use the --replace_unk flag which tells the program to copy unknown words from the input to the output, except for the pts set of experiments where it is replaced by the --phrase_table flag that uses the lexicon. we report three baselines: (1) one model trained on clear only, (2) another model trained on wiki-large fr only, and (3) the identity baseline which corresponds to the case where the output is the copy of the input. ## evaluation we evaluate the models using several metrics: (1) bleu @xcite , initially designed for the evaluation in machine translation, is also used in text simplification which can be seen as a monolingual translation task. it compares the system output with the reference data. this metric gives a rough indication of the performance of a system, especially regarding grammaticality and meaning preservation, but it is not a strong indicator for simplification @xcite ; (2) sari @xcite is currently considered as the most common metric for text simplification. sari is computed by comparing the system output against the reference, and against the input as well. it should be noted that sari is more reliable when several references are available @xcite , which is not the case in our experiments; (3) kandel @xcite is a readability metric. it does not compare the output with the reference or the input, and only provides formal indicators such as sentence length and number of syllables per word. it is an adaptation of the flesch @xcite readability measure -that was designed for english -to the french language. the absolute indexes are not informative by themselves: the measure is described to be more relevant for comparisons. higher scores mean that the text should be easier to read. we computed the first two metrics with the easse evaluation suite for automatic text simplification @xcite . in as for clear ( ## conclusion we addressed the biomedical text simplification in french, which is, to our knowledge, the first attempt to perform this task with a machine translation technique. in order to cope with the lack of french data for simplification, we translated to french a freely available english corpus. we made the translation available, as well as the parallel data from clear foot_0 . using those data we achieved improvements over the baselines for french biomedical text simplification. indeed, baselines produce incorrect and imperfect simplifications, but the results are significantly improved with large datasets. we propose several experiments that indicate that (1) an automatically translated resource can help in a low resource setting, (2) a small amount of good quality specialized data can significantly improve overall performances, (3) multiword units processing can be dealt with by adding a lexicon to the training set. that is to say that mixing data of different linguistic nature helps simplification. overall, we can obtain interesting simplification results which often prove to be more creative than the reference and yet correct. besides, the impact of large native data for simplification should also be studied.