Evaluation of Machine Translation Methods applied to Medical Terminologies (2020)

Abstract: Medical terminologies resources and standards play vital roles in clinical data exchanges, enabling significantly the services’ interoperability within healthcare national information networks. Health and medical science are constantly evolving causing requirements to advance the terminologies editions. In this paper, we present our evaluation work of the latest machine translation techniques addressing medical terminologies. Experiments have been conducted leveraging selected statistical and neural machine translation methods. The devised procedure is tested on a validated sample of ICD-11 and ICF terminologies from English to French with promising results.

## introduction medical terminologies are of essential importance for health institutions to store, organize and exchange all medical-related data generated in labs, hospitals and other healthcare entities. they are arranged systematically in dictionaries and lexicons, that follow specific structures and coding rules. in order to facilitate hierarchies and connections, the terms are represented by ontologies, enabling us to keep additional information (e.g. a family of diseases). who international classification of diseases (icd) foot_0 terminology is a diagnostic classification standard for epidemiology, clinical and research purposes. it is the most used medical dictionary across national health organizations worldwide. who is responsible to maintain the icd editions for the english language. icd-11 is the latest edition, adopted on may 25th, 2019. as the initial medical lexicons which contain these ontologies are created in english, there is an evident need for translation in other languages. this translation process can be expensive both in terms of time and resources, while the vocabulary and number of medical terms can reach high numbers and require health professional efforts for evaluation. this work constitutes a generic, languageindependent and open methodology for medical terminology translation. to illustrate our approach, which is based on automated machine translation methods, we will attempt to develop a first baseline translation from english to french for the icd-11 classification. we also test on the international classification of functioning, disability and health (icf) terminology foot_1 . first, we are going to investigate existing machine translation research studies concerning medical terms and documents, with a comparison of the relative methods. next, we present our proposed methodology. afterwards, we show our experiments and results. last, we conclude with recommendations for future work. ## related work translating medical terminologies has been a wellstudied topic, with many approaches coming from machine translation. traditional machine translation models first incorporated statistical models, whose parameters are set through the analysis of bilingual text corpora. @xcite investigated the usefulness of a large medical database (the unified medical language system) for the translation of dialogues between doctors and patients using a statistical machine translation system. they showed that the extraction of a large dictionary and the usage of semantic type information to generalize the training data significantly improves the translation performance. @xcite presented a method to automatically translate a large class of terms in the biomedical domain from one language to another; it is evaluated on translations between french and english. their technique relies on a supervised machine-learning algorithm, called os-tia @xcite , that infers transducers from examples of bilingual term-pairs. such transducers, when given a new term in english (respectively french), must propose the corresponding french (resp. english) term. ## methodology in the following section we describe the steps of our research methodology. first, a brief description of the terminologies and other corpora utilized is shown. next, we describe the tools and libraries we have experimented with. finally, the translation pipeline is presented. ## experiments & results in this section, we present the conducted experiments and obtained results. we selected two toolkits, due to their popularity and efficiency. moses represents the smt tools, and fairseq represents the nmt domain. the summarized results of our experiments are visualized in ## conclusion in this work, an automated pipeline for translating and evaluating medical terminologies is presented. the pipeline is tested comparing different machine translation methods, to translate who icd-11 and icf terminologies from english to french. over ten legacy medical terminologies along with icd-10 are used for training the pipeline. a traditional moses smt approach that manages to produce a good baseline translation is shown. we have tested nmt methods and found that finetuning largely pre-trained models like fairseq's cnn on medical terminologies, incorporating transfer learning, can improve the quality of the translation. last, we presented a simple method to generate automatically a test subset via existing terminologies. the pipeline is adaptive to the typology of the studied terminology and it can be extrapolated easily to other languages for medical terminologies. the methodology enables researchers and healthcare end-users globally with a jump start approach that allows fast and effective translation of newly updated versions of terminologies. for future work, using multilingual models @xcite ) may omit the need for training multiple models in different languages. last, additional medical datasets can be explored, not only for training but for creating larger validated corpora as well, following the constantly growing area of freely available language resources.