Segment-based Interactive Machine Translation at a Character Level (2023)

Abstract: To produce high quality translations, human translators need to review and correct machine translation hypothesis in what it is known as post-editing. In order to reduce the human effort of this process, interactive machine translation proposed a collaborative framework in which human and machine work together to generate the translations. Among the many protocols proposed throughout the years, the segment-based one established a paradigm in which the post-editor was allowed to validate correct word sequences from a translation hypothesis and introduced a word correction to help the system improve the next hypothesis. In this work we propose an extension to this protocol: instead of having to the type the complete word correction, the system will complete the user’s correction while they are typing. We evaluated our proposal under a simulated environment, achieving a significant reduction of the human effort.

## introduction the machine translation (mt) field has significantly changed over the last few years due to the appearance and application of neural models. thanks to this emergent technology, researchers have been able to accomplish human parity in several mtrelated tasks @xcite . thus, in the future we might no longer need human translators to review and correct translations hypothesis from an mt system to achieve high-quality translations. until this future arrives, human experts need to be involved in the translation process and post-edit the mt system's output in order to get translations of the required high quality. to alleviate the cost of the post-editing task, interactive machine translation (imt) proposed a collaborative framework in which human and machine work together to construct the final translation: instead of correcting the complete translation hypothesis, the expert can provide the system with some feedback which it uses to generate a new hypothesis. this process is repeated until the user is satisfied with the system's hypothesis. among the different protocols proposed in the literature, we find segment-based imt @xcite . in this paradigm, the user reviews the system's translation hypothesis and can validate sequences of words which they consider to be correct. then, they make a word correction. the system reacts to this feedback by generating a new hypothesis and, thus, starting a new iteration of the process. figure 1 illustrates an iteration of a segmentbased imt session where the user has to translate a sentence from spanish to english. given the hypothesis generated by the mt model, the user starts validating a sequence of correctly translated segments and types the word first to help the system fulfill the sequence of words between the first two validated segments. the system generates a new hypothesis with the feedback from the validated segments and the word correction. the process that describes the figure is repeated until the hypothesis generated by the system is good enough that the user validates it. in this work, we propose to extend this protocol so that instead of having to make a word correction, the system generates a new hypothesis as soon as indiana was the sooner state to impose that condition. target: indiana was the first state to impose such a requirement. source: el estado de indiana fue el primero en exigirlo. first such a condition. first indiana was the state to impose the user starts typing, helping them complete the correction and, thus, reducing even more the typing effort. ## related work reducing the effort users need to perform during the translation process is a problem that has been in the spotlight of imt researchers since its paradigm was proposed as an alternative to post-editing @xcite . in this first approach, the user selected a section of the source text and started to type its translation. when the user typed a character, the system displayed a list of possible words that the user might accept or reject. since then, researchers have studied various approaches to reduce the user effort even more. over time, appeared projects such as transtype @xcite , matecat @xcite , casmacat @xcite , and transmart @xcite , whose aim was to create a workbench with an array of innovative features that were not available in other tools at their start. adding multiple ways to edit a translation and to visualize the information helped to reduce the effort. among the features that each workbench integrated, they found helpful to use an imt system to predict either the current word or the rest of the translation. these projects used the prefix-based protocol introduced by @xcite . in this protocol, the user reviews the system's translation hypothesis from left to right, validating one segment from the start of the translation until it finds the first error to correct. the user validates a larger prefix at each iteration, and the system produces an appropriate suffix for completing the translation. the protocol has evolved over the years, presenting advances related to suffix generation @xcite @xcite , introducing new kinds of interaction @xcite , and visualization of the information with confidence measures @xcite . the segment-based protocol, introduced by @xcite , has also evolved over the years, applying over it techniques from other mt subfields. researchers have used reinforcement learning @xcite and confidence measures @xcite to obtain the validated segments and improve segment prediction with text-infilling methods @xcite . in this work, we extend the segment-based protocol from typing the whole word to perform a new prediction to only needing to type one character. this same approach has also been studied for the prefix-based protocol @xcite @xcite . ## segment-based imt in the segment-based imt framework, a human translator and an mt system work together to create high-quality translations. this collaboration starts with the system proposing an initial translation hypothesis y i 1 of length i. the user, then, reviews this hypothesis and validates those sequences of words which they consider to be correct ( f1 , . . . , fn ; where n is the number of non-overlapping validated segments). next, they are able to merge two consecutive segments fi , fi+1 into a new one. finally, they make a word correction-introducing a new one-word validated segment, fi , which is inserted in f n 1 . in response to this user feedback, the system generates a sequence of new translation segments g n 1 = g 1 , . . . , g n ; where each g n is a subsequence of words in the target language. this sequence complements the user's feedback to conform the new hypothesis: @xcite formalized the word probability expression for the words belonging to a validated segment fn as: ## experimental framework this section presents the details of our experimental session. we start by presenting the evaluation metrics used for assessing our proposal. then, we describe the corpora used for training our models. after that, we detail the training procedure of our indiana was the sooner state to impose that condition. target: indiana was the first state to impose such a requirement. source: el estado de indiana fue el primero en exigirlo. f f irst such a condition. indiana was the state to impose mt systems. finally, we describe how we performed the user simulation. ## results in order to assess our proposal, we evaluated the segment-based imt protocol at word and character level. we aim to see in the character-level experiments a reduction in the ksr and ksmr due to letting the system try to autocomplete the wrong word instead of typing it manually. the translation tasks europarl and eu have a higher reduction factor of the ksr. we can deduce that this is due to these corpora having a larger vocabulary, which helps the system to find partially correct words avoiding the worst-case scenario of correcting a word character by character. moreover, the use of bpe also assists the character level approach, since even if the model does not know the correct word, it is able to predict some of its sub-words correctly. this high reduction in the ksr is the expected behavior, given that the mt models are good enough to predict correctly the desired word with just a few characters. even in the worst-case scenario, the system can never correct an error with just a subset of its characters; the ksr maintains the same, as the user needs to type all the characters to rectify the error in both cases. however, working at the character level supposes a minor increment in the mar because if the next character to correct is not adjacent to the previous one, the user has to move the cursor to the new position. when working at a word level, each word supposes only one mouse action while at a character level each could add multiple mouse actions. word-level approach source: el estado de indiana fue el primero en exigirlo. target: indiana was the first state to impose such a requirement. ## conclusions and future work in this work we have extended the segment-based imt protocol so that the system also helps the user through the word correction step of the process. now, instead of having to input the whole word, the system offers suggestions while the user is typing the correction. we assessed our proposal under a simulated environment, observing a significant reduction of the overall human effort. as a future work we would like to extend this feature by providing the user with a list of suggested words, instead of just auto-completing the word correction with only the most probable one. additionally, we would like to conduct a user evaluation to better assess the impact of our proposal, taking also into consideration other factors such as time.