O pen TIPE : An Open-source Translation Framework for Interactive Post-Editing Research (2023)

Abstract: Despite the latest improvements on machine translation, professional translators still must review and post-edit the automatic output to ensure high-quality translations. The research on automating this process lacks an interactive post-editing environment implemented for this purpose; therefore, current approaches do not consider the human interactions that occur in real post-editing scenarios. To address this issue, we present OpenTIPE, a flexible and extensible framework that aims at supporting research on interactive post-editing. Specifically, the interactive environment of OpenTIPE allows researchers to explore human-centered approaches for the post-editing task. We release the OpenTIPE source code and showcase its main functionalities with a demonstration video and an online live demo.

## introduction recent advances in machine translation (mt) and the adoption of neural architectures (bentivogli et al., 2016) led to significant improvements in translation quality aspects such as fluency or adequacy (bentivogli et al., 2016; bojar et al., 2017). despite these advancements, mt is not yet on a par with human performance (läubli et al., 2018; toral, 2020) and human post-editors need to edit the mt output to obtain high-quality translations that also adapt to a specific domain and style. to reduce the manual efforts, the research community proposed to automate this process and implemented automatic post-editing (ape) models that automatically learn post-editing rules from revised translations (do carmo et al., 2020). the use of these models is specially indicated in environments where the mt system that generates the translations is not accessible for retraining (chatterjee et al., 2015). to date, the automatic corrections generated by the state-of-the-art ape models still require proofreading, so there is no solution that fully automates the translation process. in fact, the post-editing task remains mostly manual in production workflows. instead of fully automating the post-editing task, escribe and mitkov (2021) suggest that postediting would greatly benefit from human-centered approaches that leverage the post-editor's corrections and their interactions with the translation interface. for example, ape models could improve over time by incrementally learning from human corrections (chatterjee et al., 2017). while human-computer interaction has been explored in mt with the help of translation frameworks such as casmacat (alabau et al., 2013), there is no such interactive environment designed for the post-editing task (do carmo et al., 2020; escribe and mitkov, 2021). therefore, current research in interactive post-editing (ipe) is limited to simulate the human interaction by feeding pre-existing corrections to the post-editing process sequentially (ortiz-martínez and casacuberta, 2014; chatterjee et al., 2017; negri et al., 2018a). additionally, human corrections are scarce and these approaches often rely on synthetic post-edited datasets such as escape (negri et al., 2018b). although these artificial settings enabled valuable research in this field, they lack the human intervention as in real-world post-editing scenarios. in this paper, we present opentipe, an opensource framework that enables research on postediting in an interactive environment. in particular, opentipe implements the following main features: to support the user during the post-editing process, the tool provides automatic postediting suggestions from an ape model, which can be directly applied to the revised translation (see section 3.1). to the best of our knowledge, this is the first interactive environment designed to facilitate research on ipe. we hope that opentipe fosters further research in this field that can be applied to improve the overall post-editing experience. ## related work most of the related work focuses on implementing ape approaches to automate the post-editing task. however, these approaches do not enhance humancomputer interaction. in fact, human-centered approaches have been only explored in mt settings. in this section, we first summarize the work on ape and their online approaches, which simulate the post-editor behaviour by implementing continuous streams of corrections. we then describe the research on interactive mt, which is the most inline with this work. ## the opentipe framework opentipe implements a microservice architecture, consisting of independent services that are orchestrated with docker compose. @xcite the advantages of this architecture are twofold. first, it enables separation of concerns. that is, each service implements its own function and runtime environment, reducing code complexity and errors. second, its flexibility, as services can be easily replaced. the main components of the opentipe implementation are the frontend, which provides the graphical user interface to translate and post-edit texts, and the backend services: the backend api, mt and ape services, and data storage. additionally, opentipe supports user authentication with firebase. @xcite figure 1 illustrates the overall architecture of opentipe and the interaction between the different services. in the following, we describe the technical details of each component. ## data collection and representation human post-edited data is a valuable resource to improve ape models. however, this is a scarce resource and researchers are often dependent on synthetic datasets. therefore, the collection of human data is an important aspect of this ipe environment. researchers can then leverage the data to implement human-in-the-loop approaches and assess the performance of different ape settings. our implementation of opentipe collects (1) human postedited translations and (2) user interactions with the post-editing environment and it stores them as json objects in the mongodb database (section 3.3). the rest of this section describes the representation of these data in more detail. ## usability study we perform a user study to assess the usability of the opentipe user interface. in particular, we conduct a controlled observation with a professional translator and a survey-based assessment with eight non-professional translators. the latter are academics between 21 and 30 years old (62.5% are male and 37.5% female), who indicated that they frequently use translation services, such as deepl foot_16 and google translate. @xcite while the observation-based setting allows us to get insights on the interactions of an expert with the tool, the survey-based assessment gives us a general subjective view of the user interface usability. in both settings, all participants saw the interface of opentipe for the first time during the study. we start the study explaining its purpose to the participants. in the observation-based setting, the professional translator is aware that he is being observed during the process. @xcite we then provide them with a text to translate and the following instructions: ## conclusion we presented opentipe, the first interactive framework that aims at supporting the research of humancentered approaches for post-editing. in contrast to research in machine translation, human-computer interaction has been only simulated for the postediting task, since there was no interactive environment available for this purpose. opentipe follows a microservice architecture such that it can be easily extended and adapted to other models or features. additionally, it collects human post-editing data and the user interactions with the interface. these data are key to implement human-in-the-loop approaches that learn from human corrections over time. we expect this work to foster future research on interactive approaches that enhance the performance of the post-editing process. we are excited to explore this direction in future work.