Ellipsis Translation for a Medical Speech to Speech Translation System (2020)

Abstract: In diagnostic interviews, elliptical utterances allow doctors to question patients in a more efficient and economical way. However, literal translation of such incomplete utterances is rarely possible without affecting communication. Previous studies have focused on automatic ellipsis detection and resolution, but only few specifically address the problem of automatic translation of ellipsis. In this work, we evaluate four different approaches to translate ellipsis in medical dialogues in the context of the speech to speech translation system BabelDr. We also investigate the impact of training data, using an under-sampling method and data with elliptical utterances in context. Results show that the best model is able to translate 88% of elliptical utterances.

## introduction ellipsis is one of the least studied discursive phenomena in automatic translation. like anaphora, ellipsis require context to be understood, but contrary to anaphora, there is no indicator that there is a missing part in the sentence 1 . the characterising feature of ellipsis is that "elements of semantic content are obtained in the absence of any corresponding form. the syntax thus appears to be incomplete. more specifically, the implicit semantic context is recovered from elements of linguistic c 2020 the authors. this article is licensed under a creative commons 3.0 licence, no derivative works, attribution, cc-by-nd. 1 ellipsis is "a case of anaphora, where the anaphor is a null proform (zero-anaphora)" @xcite and extralinguistic context" @xcite . in nlp, different studies have focused on automatic ellipsis detection and resolution either with rules (patterns or grammars) (for example, the pioneer work from @xcite or classification techniques (for example, @xcite @xcite @xcite . however, only few studies specifically address this problem in machine translation (mt), despite the recent interest for context modelling in neural machine translation (see for example, @xcite . very recently, some qualitative studies showed the negative impact of ellipsis on generalist neural systems (deepl, google translate, etc.) from a translation point of view in the english-french pair (for example, @xcite . in this paper, we focus on automatic translation of ellipsis in medical dialogues, in the particular context of babeldr, a speech to speech translation system for the medical domain @xcite foot_0 . elliptical utterances are very common in dialogues, since they ensure the principle of economy and provide a way to avoid duplication @xcite . in the medical dialogues we are interested in, ellipsis allows doctors to question patients in a more efficient way (where is your pain? in the back? is the pain severe? moderate?) @xcite . literal translation of these elliptical utterances is rarely possible without affecting communication, in particular with structurally different languages which do not share the same type of ellipsis. for example in japanese, adjectival ellipsis are very informal and should be translated by complete sentences @xcite . the following examples illustrate elliptical utterances where literal translation is problematic, as it produces agreement errors, wrong prepositions or other syntactical or grammatical issues that can make the elliptical utterance difficult to understand. source: is the pain intense? ->mt: la douleur est-elle intense source: sudden? -> mt: * soudain source: do you have pain in your stomach? -> mt: le duele el estómago? source: in your head? -> mt: * en la cabeza? source: is the pain severe -> mt: hageshii itami desu ka? source: moderate? -> mt: * chuuteido? the aim of this paper is to compare different approaches to translate ellipsis in the context of ba-beldr. section 2 describes the babeldr system. section 3 outlines the methodology, including the objective and research questions, the test data and the evaluation metrics. section 4 presents the approaches and models, followed by section 5 which describes the different sets of training data. section 6 presents the results and section 7 concludes. ## approaches as mentioned earlier, our objective is to use the context (previous utterance) to map elliptical utterances to the closest core sentence. figure 1 provides an overview of the ellipsis translation task as it would be performed in babeldr. starting with a source sentence, we perform ellipsis detection using a binary classifier (support-vector machine) trained on handcrafted features. in this context, elliptical sentences can easily be detected by sentence length and syntactic structure. therefore, the sentence length, the first word of the sentence and its part-of-speech are used as features to train the classifier. this method achieves 98% of accuracy on ellipsis detection. if the utterance is identified as an ellipsis, it is concatenated with the previous utterance from the dialog @xcite . this concatenated sentence is then processed like other utterances. in the following sections, we describe the four approaches applied after concatenation. the same training data (described in section 5) was used for all approaches. the source sentences were preprocessed using the same method for all the models: they have been lower cased and tokenized. each approach has its own built-in tokenization method to reach optimal results, except for machine translation where we applied bpe. ## training data in this section, we describe the training data sets used for this study. all data were generated 6 https://opennmt.net/opennmt-tf/model.html from a recent version of the babeldr scfg for the abdominal diagnostic domain and consist of variation-core pairs. ## results in this section we first describe the evaluation of the under-sampling method (subsection 6.1). we then give results for different models trained with under-sampled data (subsection 6.2). finally, including only the best model for each approach in terms of f1, we evaluate the impact of training on ellipsis data (subsection 6.3). ## conclusion in this study we have applied different approaches to an ellipsis translation task, in the context of a medical speech translator. we have also experimented with different forms of training data generated from the babeldr scfg. results show that under-sampling the training data improves results for all tested approaches. of all the tested systems, the hybrid approach, combining neural machine translation and classification models is the most successful both in terms of our task specific metric (ser) and in terms of precision/recall/f1. we also observe that the inclusion of ellipsis training data further improves results. one limitation of this study is the annotation of the test data. each source variation has been annotated with a single correct core sentence, but this does not reflect the real use case: the purpose of babeldr is to allow doctors to collect information from the patient, not to translate their exact utterance. often, even if the core sentence is not an exact match (e.g. ""in the lower part" vs "in the lower part of the abdomen"), in context it still allows the doctor to obtain the required information. in future work, a more task-oriented annotation approach would be interesting. a further aspect worth investigating is exploring novel architectures to add the context in different ways: train a context aware decoder to correct translations @xcite , for neural machine translation, ) or train a dual-source bert @xcite adding context on the tuning step for sequence classification. finally, future work will also include the replication of these experiments with data from real diagnostic interviews and with data from other diagnostic domains.