Word Sense Disambiguation for Automatic Translation of Medical Dialogues into Pictographs (2023)

Abstract: Word sense disambiguation is an NLP task embedded in different applications. We propose to evaluate its contribution to the automatic translation of French texts into pictographs, in the context of communication between doctors and patients with an intellectual disability. Different general and/or medical language models (Word2Vec, fastText, CamemBERT, FlauBERT, DrBERT, and CamemBERT-bio) are tested in order to choose semantically correct pictographs leveraging the synsets in the French WordNets (WOLF and WoNeF). The results of our automatic evaluations show that our method based on Word2Vec and fastText significantly improves the precision of medical translations into pictographs. We also present an evaluation corpus adapted to this task.

## introduction dialogue between doctors and patients is essential, as it enhances the patients' health status, their medication adherence, and their overall quality of life @xcite . however, this dialogue can be impaired by misunderstandings, in particular for patients with an intellectual disability (id). various augmentative and alternative communication (aac) systems are used by people with disabilities @xcite , including automatic translation tools from text into pictographs @xcite . one of the main issues that those systems face is polysemy. for example, in the french sentence to be translated "avez-vous appliqué une crème sur la lésion ?" (did you put cream on the lesion?), "crème" (cream) can be interpreted as ointment or liquid cream. a translation system has to be able to produce the correct pictograph, here one that would represent @xcite in this article, we focus on word sense disambiguation (wsd) of french polysemous words that can be used orally by doctors in questions and instructions for anamnesis in emergency settings @xcite . the text-to-picto system we use translates french into arasaac, foot_0 sclera foot_1 or beta foot_2 pictograph sets, designed for aac users with an id @xcite . in order to provide a better semantic understanding of the input sentence, we test various language models (static, contextual, trained on general and/or medical data), and different french sense inventories. in addition, we present an evaluation corpus adapted to this task. section 2 describes existing work on wsd and text-to-pictograph systems. section 3 introduces our methodology and the language models we used, while section 4 presents the text-to-picto system, the evaluation corpus, and the results. our evaluations with word2vec and fasttext show significant improvements over the baseline with the text-to-picto tool. we discuss the results in section 5. ## related work wsd has already been used in automatic text-topictograph systems, in order to improve the translation of polysemous words for the general language. for english, @xcite describe a basic wsd tool based on wordnet @xcite , but they do not evaluate its effectiveness within their text-to-pictograph translation system. @xcite test different wsd techniques -original lesk, adapted lesk, max similarity, support vector machine (svm) -with the english word-net. they show that the system with the svm obtains the best results @xcite . in text-to-picto, a system originally designed for dutch @xcite , @xcite use an external wsd tool, based on svm and developed within the framework of the dutchsemcor project @xcite . @xcite specifically evaluated the contribution of this wsd tool using a corpus of 50 sentences that contain at least one ambiguous word. she obtained an improvement in precision for sclera pictographs (from 29/50 to 41/50), and for beta (from 28/50 to 42/50), demonstrating the added value of integrating a step of wsd. for french, @xcite ; @xcite were the first to underline the importance of using wsd in a pictograph translation tool. related to medical language, there are translation systems with pictographs, but they do not include wsd. this is the case of the french text-to-picto @xcite , but also for pictodr, based on a neural translation approach using concepts, instead of words @xcite . we therefore aim to assess the contribution of wsd in the context of specialized language for automatic translation into pictographs, an issue that has not yet been addressed in the literature. ## methodology we present our wsd algorithm below, through the example of the noun "alcool" (alcohol) to be disambiguated (figure 1 ) in the sentence "avezvous bu de l'alcool ?" (did you drink alcohol?). the two possible translations into an arasaac pictograph are: alcoholic drink, and isopropyl alcohol. the lemma "alcool" refers to three different synsets in wolf @xcite , the french wordnet used by default in the text-to-picto system @xcite . we differentiate steps using static embeddings and contextual embeddings by marking them respectively with (a) and (b). example: {'synset1' (07884567-n): 0.64, 'synset2' (14708720-n): 0.35, 'synset3' (14941230-n): 0.25} 4. use the cosine scores to select the pictograph(s) to retrieve. we rank the synsets, sorted by cosine similarity in descending order. we start by retrieving the pictograph(s) of the synset that comes first (rank 1), if this synset is not linked to a pictograph, we retrieve the pictograph(s) of the synset that comes after, and so on until a pictograph is found (@xmath0. example: {'synset1' (07884567-n): 26626, 'synset2' (14708720-n): 2984, 'synset3' (14941230-n): -} for our language models, we used pre-trained models for french ( we also trained 500-dimension word2vec and fasttext models -cbow and skip-gram -, on the clear corpus @xcite ), foot_11 using the same word2vec hyperparameters as car-don (2021, p. 47). 13 for training with fasttext, we used the default hyperparameters. clear is a french medical corpus consisting of three sub-corpora: articles from online encyclopedias (wikipedia and vikidia), drug leaflets, and summaries of the cochrane foundation's medical scientific literature. it is a comparable corpus, with texts in a technical version and in a simple/simplified version. we used the three sub-corpora, once with medical encyclopedia articles (146 million words in total) and another time adding general articles (+65 million words). we did not pre-process this corpus before training. note that clear is a part of the nachos and biomed-fr corpora that were used to train drbert and camembert-bio. we therefore propose to evaluate several language models, trained with general and/or medical data ( ## evaluation in this section we describe our baseline -i.e., the pictograph translation tool without wsd -(section 4.1), our evaluation corpus (section 4.2), and the results (section 4.3). ## discussion we evaluated the impact of different training corpora (e.g. general language, medical language, etc.) on our performance. models pre-trained on general language data obtain higher precision on average (0.64 for the frwac2vec and frwiki2vec models, and 0.66 for fasttext trained on common crawl + wikipedia) than clear models (medical + general: 0.58; medical: 0.61). beyond the effect of the size of the training corpus of these pre-trained models, another explanation for these counter-intuitive results may be the fact that even if the words to be disambiguated are integrated into medical dialogues, they are not all medical terms: out of our 100 polysemous words, only 19% are found in the medical wiktionary extracted by @xcite , 56% in the medical lexicon of @xcite , and 29% in the snomed international terminology @xcite ; 37% of them in our corpus are in at least two of these three resources. there were two configurations for the sentence vector (step 1b): a) lemmas of the content words (e.g., "avoir boire alcool"); b) the whole sentence ("avez-vous bu de l'alcool ?"). for the relation vector ( step 2b ), we tested six configurations: a) words of the 4 types of relations; b) words of the 4 types of relations, each followed by a period; c) the usages; d) the usages followed by a period; e) the usages followed by a period and synonyms, foot_17 each followed by a period; f) the usages followed by a period and words of the 4 types of relations, each followed by a period. the sentences can be useful for bert contextual models to improve the precision (a/b-c°/d°). we have however noted that encoding only usages as relation vectors is not efficient, because not enough of them are associated with synsets linked to arasaac pictographs (recall: 0.66). therefore, usages must be combined with synonyms (b-e). bert language models applied to the wolf data with our method, however, do not offer a great improvement in precision if we compare them to the text-to-picto system in za mode, which obtains 0.45 on the same 100 polysemous words (see finally, we compared several french wordnets (see figure 5 ). each of them produced rather different pictographs, due to different synset scopes. @xcite already showed that better results can be reached with wolf than with the three versions of wonef using the text-to-picto system for the arasaac pictograph set. in wolf and two versions of wonef ("coverage" and "fscore"), only half of the english wordnet synsets have been translated into french. choosing an appropriate synset is not always enough to get a correct translation. it would also be necessary to refine the selection of pictographs within the synset obtained with wsd. this is the case of arasaac where many pictographs -sometimes twenty -can be associated with a single synset. they can be identical pictographs (with a character who is non-gendered, male or female), but also with a more or less different meaning although they belong to the same synset (e.g. "lift" the toilet seat, a baby, an object, etc.). we do not have information about the method used by arasaac to label the pictographs. ## conclusion in this paper, we performed experiments for wsd with different language models, either static (word2vec, fasttext), or contextual (camembert, flaubert, drbert, camembert-bio), in medical french. we observed that the most promising method is to use word2vec or fasttext in order to improve the precision of translations into pictographs (see