Identification of complex words and passages in medical documents in F rench (2022)

Abstract: Identification de mots et passages difficiles dans les documents médicaux en français. L’objectif de la simplification automatique des textes consiste à fournir une nouvelle version de documents qui devient plus facile à comprendre pour une population donnée ou plus facile à traiter par d’autres applications du TAL. Cependant, avant d’effectuer la simplification, il est important de savoir ce qu’il faut simplifier exactement dans les documents. En effet, même dans les documents techniques et spécialisés, il n’est pas nécessaire de tout simplifier mais juste les segments qui présentent des difficultés de compréhension. Il s’agit typiquement de la tâche d’identification de mots complexes : effectuer le diagnostic de difficulté d’un document donné pour y détecter les mots et passages complexes. Nous proposons de travail sur l’identification de mots et passages complexes dans les documents biomédicaux en français.

## introduction the purpose of automatic text simplification @xcite is to provide a new version of documents that are easier to understand by a given population @xcite @xcite @xcite or easier to process by nlp applications @xcite @xcite @xcite ). yet, it is important to know what should be simplified exactly within the documents before the simplification is done. indeed, even in technical and specialized documents, it is unnecessary to simplify everything but just those segments that present understanding difficulty. for example, the following sentence from a biomedical document contains difficult words such as (opa (acute pulmonary edema), résolutif (resolvent), vni (niv), oxygénothérapie (oxygen therapy)) which, if simplified or explained, could make the text more understandable. le patient est donc transféré en réanimation : l'oap est résolutif sous vni et oxygénothérapie. (hence, the patient is transferred to intensive care : acute pulmonary edema is resolvent with niv and oxygen therapy.) generally, biomedical documents are often considered one of the most challenging texts to read and understand by a large population. the difficulty is mainly on a lexical level since the vocabulary of medical texts are very specific and because of the document's rich terminological status. therefore, the purpose complex word identification model (cwi) is to make the diagnosis of the difficulty within a given document in order to detect words or passages within it. in our work, we propose to address the issue of identifying complex words and passages within biomedical documents in french. we first present some related work (section 2). we then introduce the reference data and our approach (section 3) and present the results obtained (section 4). finally, we conclude in section 5. 1 ## related work complex word identification has attracted the attention of researchers recently, either as part of text simplification systems @xcite or as an independent task, such as promoted by semeval 2016 @xcite , 2018 cwi shared task @xcite , or semeval 2021 @xcite . the first complex word identification semeval was introduced in 2016 to identify difficult words in an english corpus. the task was a binary classification task in which the model had to decide whether the target word was difficult or not for non-native speakers @xcite . participants exploited different classifiers such as support vector machine (svm), decision tree, random forest, logistic regression and recurrent neural network (rnn) with features like word embeddings, lexical, morphological, and psycholinguistic properties of the target words and pos tags. the participating systems showed an accuracy between 0.465 and 0.922. in 2018, the cwi shared task extended its purpose to the identification of difficult words in different languages (english, german, french, spanish), as well as in a multilingual corpus @xcite . two tasks were proposed : binary classification task (to decide whether the word is difficult or not) and probabilistic classification task (to assign a probability to a given word as being difficult). participants used different classifiers (e.g., svm, naive bayes, random forest) and different combinations of features such as word frequency, semantics, lexical, morphological, and psycholinguistic properties. in addition, some participants started to use the context of target words and word embeddings. the results of the binary classification task, f-measures were between 0.176 and 0.874 for the english corpus and between 0.577 and 0.745 for other languages. the purpose of the second semeval 2021 task was also to identify difficult words in texts from different genres in english : european parliament, bible and biomedical texts @xcite . two main differences from previous tasks : words are annotated using a 5-point likert scale (from very easy to very difficult) and consideration of polylexical units. participants used language models, such as bert @xcite or roberta @xcite , and gradient boosted regression. the evaluation metric was pearson's correlation and the participants obtained scores between 0.7886 and -0.0272 for the processing of single words and scores between 0.8612 and 0.1860 for the processing of polylexical units. besides challenge papers, @xcite proposed an approach to predict the lexical complexity of french words for non-native learners. the authors use svm with different features (e.g., word length, number of phonemes, number of syllables, phoneme/spelling coherence). they obtain accuracy between 43% and 63%. in another work, three methods for the detection of difficult words in general language english corpora are evaluated @xcite : (1) simplify everything, (2) exploit frequency using reference corpus, (3) train svm model. the svm-based method shows the highest performance with 0.771 precision, while the simplifying everything method has 0.738 precision and the frequency-based method has 0.709 precision. the frequency threshold is exploited in another work for the detection of familiarity with medical terms @xcite . the experience obtains 0.196 mean absolute error and 0.293 root mean square error. yet, another way to determine word complexity is based on the rarity of words : the words that are not found in different lexica are considered as difficult @xcite . this method shows 92% accuracy. @xcite has introduced a convolutional neural network (cnn) approach that combines different features to identify the complex word(s) in a sentence. the model uses glove word embedding @xcite feature along with other features such as word frequency, word length, number of syllables, number of vowels, tfidf, part-of-speech, dependency, and stop word. the model performs quite well on three datasets : english, german, and spanish. however, there was no french language included in the experiments. ## experiments in this section, we present the dataset, the preprocessing steps, and the models we propose in this paper. the goal of the experiments is to classify the target text in a sentence as complex or not complex and evaluate it based on the manual annotation of the dataset. ## results and discussion we started with the cnn model trained with only fasttext word embedding, and then we kept adding more features one by one. we could see that the result was improved each time we added a new feature. then, we tried removing the embedding feature, and the result dropped significantly. next, we tried bert embedding (camembert), and the result was way better than all models with fasttext. it can be due to the fact that bert embedding has captured better information on words than fasttext. next, we combined fasttext with bert ; as a result, the cnn model performs worse in comparison with all the models, the results show that in most cases the cnn model performs better than catboost and xgboost, except in the model without the word embedding, this could be an indication that catboost and xgboost learn better than the cnn model when having less information. ## conclusion in this paper, we have proposed three classifiers based on cnn, catboost, and xgboost with different feature sets such as fasttext, bert, word length, word syllable, vowel count, word rank, tfidf, langgen frequency, and clear frequency to detect complex words in french biomedical documents. we have also created the dataset and provided the set of experiments for the evaluation. the results have shown that our models perform quite well, and the cnn model overtakes the others.