MTPE quality evaluation in translator education: the postedit.me app (2024)

Abstract: This article presents the main functionality of the postedit.me app. Postedit.me is a software program that supports machine translation post-editing training in translator education, with special emphasis on standardized quality evaluation of post-edited texts produced by students. The app is made freely available to universities for teaching and research purposes.

## background translation curricula across the globe are regularly updated to reflect the most recent technological advances in the language services industry. machine translation post-editing (mtpe) is a case in point. while the first concrete proposals to incorporate mtpe training into translator education emerged in the 2010s, many translation curricula nowadays feature mtpe training in the form of stand-alone technology modules or language-pair-specific practical courses. surprisingly, however, there are relatively few digital tools that support mtpe training and learning. examples include mateo @xcite , a web interface devoted to machine translation (mt) evaluation training, and copeco @xcite , an online collaborative platform designed to collect an annotated student post-editing corpus. in this article, we describe the postedit.me app. the tool streamlines assignment of mtpe tasks to students and automates standardized quality evaluation of students' post-edited texts. ## basic functionality the postedit.me app consists of a teacher interface and a student interface. its main objective is to help mtpe trainers assess the quality of the post-edited texts produced by their students, relying on standardized annotation taxonomies. students are asked to carry out their mtpe assignments in dedicated computer-aided translation (cat) tools, and to use postedit.me to submit their final postedited texts, consult their trainer's feedback and track their progress across tasks. ## technologies used postedit.me is a server application with a web interface that runs on any system supporting docker containers, primarily modern linux. the technology stack of the app consists of several components which are all interfaced by python 3. the interface of the application is implemented in djangoframework. the app benefits especially from the built-in object-relational mapper model and the ability to easily generate forms from data models. for handling asynchronous tasks (text alignment, pos tagging), the celery library is used with rabbitmq broker. for mt and mtpe annotation, the labelstudio library is used. the app uses spacy with appropriate pre-trained models for generating pos tags and lemmas. the text alignment is quite innovative as it supports alignment of three language versions. for this reason, it mostly uses original, dedicated code, although it also calls on a few methods from sentence_transformers and nltk packages. finally, the annotated and aligned texts are stored in a custom xml format, which is processed by the beautifulsoup library. the search function is based on elasticsearch, where a custom token mapping is defined, which stores a token context (n neighboring tokens) together with each token from the text. while this solution is heavy on data storage, it maximizes the speed of the concordance and supports multiargument queries (e.g. a specific pos followed by a specific word with a specific annotation). loadbalancing is currently achieved by distributing computation-heavy tasks (alignment, pos tagging) of individual texts between computer cores. ## app availability and licensing the postedit.me app is made freely available to universities for teaching and internal research purposes. a custom licence agreement must be signed by both parties before the code, documentation and user guides are shared. on-demand custom commercial licences can be prepared for other partners, depending on the particular needs of interested parties (e.g. interoperability with specific cat tools).