pwc-archive/papers-with-abstracts huggingface dataset:
A User Study of the Incremental Learning in NMT
Authors: Miguel Domingo, Mercedes García-Martínez, Álvaro Peris, Alexandre Helle, Amando Estela, Laurent Bié, Francisco Casacuberta, Manuel Herranz
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Incremental Learning, Machine Translation, NMT, Translation

In the translation industry, human experts usually supervise and post-edit machine translation hypotheses. Adaptive neural machine translation systems, able to incrementally update the underlying models under an online learning regime, have been proven to be useful to improve the efficiency of this workflow. However, this incremental adaptation is somewhat unstable, and it may lead to undesirable side effects. One of them is the sporadic appearance of made-up words, as a byproduct of an erroneous application of subword segmentation techniques. In this work, we extend previous studies on on-the-fly adaptation of neural machine translation systems. We perform a user study involving professional, experienced post-editors, delving deeper on the aforementioned problems. Results show that adaptive systems were able to learn how to generate the correct translation for task-specific terms, resulting in an improvement of the user’s productivity. We also observed a close similitude, in terms of morphology, between made-up words and the words that were expected.
NICE: Neural Integrated Custom Engines
Authors: Daniel Marín Buj, Daniel Ibáñez García, Zuzanna Parcheta, Francisco Casacuberta
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Machine Translation, Translation

In this paper, we present a machine translation system implemented by the Translation Centre for the Bodies of the European Union (CdT). The main goal of this project is to create domain-specific machine translation engines in order to support machine translation services and applications to the Translation Centre’s clients. In this article, we explain the entire implementation process of NICE: Neural Integrated Custom Engines. We describe the problems identified and the solutions provided, and present the final results for different language pairs. Finally, we describe the work that will be done on this project in the future.
Estimation vs Metrics: is QE Useful for MT Model Selection?
Authors: Anna Zaretskaya, José Conceição, Frederick Bane
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Machine Translation, Model Selection, Translation

This paper presents a case study of applying machine translation quality estimation (QE) for the purpose of machine translation (MT) engine selection. The goal is to understand how well the QE predictions correlate with several MT evaluation metrics (automatic and human). Our findings show that our industry-level QE system is not reliable enough for MT selection when the MT systems have similar performance. We suggest that QE can be used with more success for other tasks relevant for translation industry such as risk prevention.
Persistent MT on software technical documentation - a case study
Authors: María Concepción Laguardia
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11

We report on the features and current challenges of our on-going implementation of a Persistent MT workflow for Citrix Product Documentation, to increase localization coverage to 100% content in docs.citrix.com into German, French, Spanish, Japanese and Simplified Chinese. By the end of 2019, we had processed seven million words of English documentation with this model, across 24 doc sets, and raised localization coverage from 40% to 100% of the content of our documentation repositories. This has boosted our global reach across the entire Citrix portfolio (Digital Workspace, Networking, and Analytics). The current implementation requires a process of Light Post-editing (LPE) for all languages, in order to fix over-translations, out-of-domain words, inline tags and markdown errors in the raw output.
Insights from Gathering MT Productivity Metrics at Scale
Authors: Georg Kirchner
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Management, NMT, Translation

In this paper, we describe Dell EMC’s framework to automatically collect MT-related productivity metrics from a large translation supply chain over an extended period of time, the characteristics and volume of the gathered data, and the insights from analyzing the data to guide our MT strategy. Aligning tools, processes and people required decisions, concessions and contributions from Dell management, technology providers, tool implementors, LSPs and linguists to harvest data at scale over 2+ years while Dell EMC migrated from customized SMT to generic NMT and then customized NMT systems. For content in two quality tiers, we ranked language pairs by productivity, graphed trendlines, compared the time needed to edit machine translations versus fuzzy matches, studied the time spent on segments with no post-edits, and going by the post-edit density, re-viewed segment distribution on a post-edit scale of 1 to 10 and any correlation between the extent of edits and segment length.
On the differences between human translations
Authors: Maja Popovic
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Machine Translation, Sentence, Translation

Many studies have confirmed that translated texts exhibit different features than texts originally written in the given language. This work explores texts translated by different translators taking into account expertise and native language. A set of computational analyses was conducted on three language pairs, English-Croatian, German-French and English-Finnish, and the results show that each of the factors has certain influence on the features of the translated texts, especially on sentence length and lexical richness. The results also indicate that for translations used for machine translation evaluation, it is important to specify these factors, especially if comparing machine translation quality with human translation quality is involved.
Re-design of the Machine Translation Training Tool (MT3)
Authors: Paula Estrella, Emiliano Cuenca, Laura Bruno, Jonathan Mutal, Sabrina Girletti, Lise Volkart, Pierrette Bouillon
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Machine Translation, Translation

We believe that machine translation (MT) must be introduced to translation students as part of their training, in preparation for their professional life. In this paper we present a new version of the tool called MT3, which builds on and extends a joint effort undertaken by the Faculty of Languages of the University of Córdoba and Faculty of Translation and Interpreting of the University of Geneva to develop an open-source web platform to teach MT to translation students. We also report on a pilot experiment with the goal of testing the viability of using MT^3 in an MT course. The pilot let us identify areas for improvement and collect students’ feedback about the tool’s usability.
Multidimensional assessment of the eTranslation output for English–Slovene
Authors: Mateja Arnejšek, Alenka Unk
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Machine Translation, Translation

The Slovene language department of the European Commission Directorate-General for Translation has always been an early adopter of new developments in the area of machine translation. In 2018, the department started using neural machine translation produced by the eTranslation in-house engines. In 2019, a multidimensional assessment of the eTranslation output for the language combination English–Slovene was carried out. It was based on two user satisfaction surveys, an analysis of detected and reported errors and an ex post analysis of a sample. As part of the assessment effort, a categorisation of errors was devised in order to raise awareness among translators of the potential pitfalls of neural machine translation.
How do LSPs compute MT discounts? Presenting a company’s pipeline and its use
Authors: Randy Scansani, Lamis Mhedhbi
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Machine Translation, Translation

In this paper we present a pipeline developed at Acolad to test a Machine Translation (MT) engine and compute the discount to be applied when its output is used in production. Our pipeline includes three main steps where quality and productivity are measured through automatic metrics, manual evaluation, and by keeping track of editing and temporal effort during a post-editing task. Thanks to this approach, it is possible to evaluate the output quality and compute an engine-specific discount. Our test pipeline tackles the complexity of transforming productivity measurements into discounts by comparing the outcome of each of the above-mentioned steps to an estimate of the average productivity of translation from scratch. The discount is obtained by subtracting the resulting coefficient from the per-word rate. After a description of the pipeline, the paper presents its application on four engines, discussing its results and showing that our method to estimate post-editing effort through manual evaluation seems to capture the actual productivity. The pipeline relies heavily on the work of professional post-editors, with the aim of creating a mutually beneficial cooperation between users and developers.
PosEdiOn: Post-Editing Assessment in PythOn
Authors: Antoni Oliver, Sergi Alvarez, Toni Badia
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Machine Translation, NMT, Sentence, Translation

There is currently an extended use of post-editing of machine translation (PEMT) in the translation industry. This is due to the increase in the demand of translation and to the significant improvements in quality achieved by neural machine translation (NMT). PEMT has been included as part of the translation workflow because it increases translators’ productivity and it also reduces costs. Although an effective post-editing requires enough quality of the MT output, usual automatic metrics do not always correlate with post-editing effort. We describe a standalone tool designed both for industry and research that has two main purposes: collect sentence-level information from the post-editing process (e.g. post-editing time and keystrokes) and visually present multiple evaluation scores so they can be easily interpreted by a user.
Quantitative Analysis of Post-Editing Effort Indicators for NMT
Authors: Sergi Alvarez, Antoni Oliver, Toni Badia
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Machine Translation, NMT, Translation

The recent improvements in machine translation (MT) have boosted the use of post-editing (PE) in the translation industry. A new machine translation paradigm, neural machine translation (NMT), is displacing its corpus-based predecessor, statistical machine translation (SMT), in the translation workflows currently implemented because it usually increases the fluency and accuracy of the MT output. However, usual automatic measurements do not always indicate the quality of the MT output and there is still no clear correlation between PE effort and productivity. We present a quantitative analysis of different PE effort indicators for two NMT systems (transformer and seq2seq) for English-Spanish in-domain medical documents. We compare both systems and study the correlation between PE time and other scores. Results show less PE effort for the transformer NMT model and a high correlation between PE time and keystrokes.
Comparing Post-editing based on Four Editing Actions against Translating with an Auto-Complete Feature
Authors: Félix do Carmo
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Descriptive

This article describes the results of a workshop in which 50 translators tested two experimental translation interfaces, as part of a project which aimed at studying the details of editing work. In this work, editing is defined as a selection of four actions: deleting, inserting, moving and replacing words. Four texts, machine-translated from English into European Portuguese, were post-edited in four different sessions in which each translator swapped between texts and two work modes. One of the work modes involved a typical auto-complete feature, and the other was based on the four actions. The participants answered surveys before, during and after the workshop. A descriptive analysis of the answers to the surveys and of the logs recorded during the experiments was performed. The four editing actions mode is shown to be more intrusive, but to allow for more planned decisions: although they take more time in this mode, translators hesitate less and make fewer edits. The article shows the usefulness of the approach for research on the editing task.
A human evaluation of English-Irish statistical and neural machine translation
Authors: Meghan Dowling, Sheila Castilho, Joss Moorkens, Teresa Lynn, Andy Way
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Machine Translation, Translation

With official status in both Ireland and the EU, there is a need for high-quality English-Irish (EN-GA) machine translation (MT) systems which are suitable for use in a professional translation environment. While we have seen recent research on improving both statistical MT and neural MT for the EN-GA pair, the results of such systems have always been reported using automatic evaluation metrics. This paper provides the first human evaluation study of EN-GA MT using professional translators and in-domain (public administration) data for a more accurate depiction of the translation quality available via MT.
Machine Translation Quality: A comparative evaluation of SMT, NMT and tailored-NMT outputs
Authors: Maria Stasimioti, Vilelmini Sosoni, Katia Kermanidis, Despoina Mouratidis
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Machine Translation, NMT, Translation

The present study aims to compare three systems: a generic statistical machine translation (SMT), a generic neural machine translation (NMT) and a tailored-NMT system focusing on the English to Greek language pair. The comparison is carried out following a mixed-methods approach, i.e. automatic metrics, as well as side-by-side ranking, adequacy and fluency rating, measurement of actual post editing (PE) effort and human error analysis performed by 16 postgraduate Translation students. The findings reveal a higher score for both the generic NMT and the tailored-NMT outputs as regards automatic metrics and human evaluation metrics, with the tailored-NMT output faring even better than the generic NMT output.
QE Viewer: an Open-Source Tool for Visualization of Machine Translation Quality Estimation Results
Authors: Felipe Soares, Anna Zaretskaya, Diego Bartolome
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Machine Translation, Sentence, Translation

QE Viewer is a web-based tool for visualizing results of a Machine Translation Quality Estimation (QE) system. It allows users to see information on the predicted post-editing distance (PED) for a given file or sentence, and highlighted words that were predicted to contain MT errors. The tool can be used in a variety of academic, educational and commercial scenarios.
Document-Level Machine Translation Evaluation Project: Methodology, Effort and Inter-Annotator Agreement
Authors: Sheila Castilho
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Document Level Machine Translation, Machine Translation, Sentence, Translation

Document-level (doc-level) human eval-uation of machine translation (MT) has raised interest in the community after a fewattempts have disproved claims of “human parity” (Toral et al., 2018; Laubli et al.,2018). However, little is known about bestpractices regarding doc-level human evalu-ation. The goal of this project is to identifywhich methodologies better cope with i)the current state-of-the-art (SOTA) humanmetrics, ii) a possible complexity when as-signing a single score to a text consisted of‘good’ and ‘bad’ sentences, iii) a possibletiredness bias in doc-level set-ups, and iv)the difference in inter-annotator agreement(IAA) between sentence and doc-level set-ups.
Sockeye 2: A Toolkit for Neural Machine Translation
Authors: Felix Hieber, Tobias Domhan, Michael Denkowski, David Vilar
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Machine Translation, NMT, Translation

We present Sockeye 2, a modernized and streamlined version of the Sockeye neural machine translation (NMT) toolkit. New features include a simplified code base through the use of MXNet’s Gluon API, a focus on state of the art model architectures, and distributed mixed precision training. These improvements result in faster training and inference, higher automatic metric scores, and a shorter path from research to production.
CEF Data Marketplace: Powering a Long-term Supply of Language Data
Authors: Amir Kamran, Dace Dzeguze, Jaap van der Meer, Milica Panic, Alessandro Cattelan, Daniele Patrioli, Luisa Bentivogli, Marco Turchi
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Machine Translation, Translation

We describe the CEF Data Marketplace project, which focuses on the development of a trading platform of translation data for language professionals: translators, machine translation (MT) developers, language service providers (LSPs), translation buyers and government bodies. The CEF Data Marketplace platform will be designed and built to manage and trade data for all languages and domains. This project will open a continuous and longterm supply of language data for MT and other machine learning applications.
QRev: Machine Translation of User Reviews: What Influences the Translation Quality?
Authors: Maja Popovic
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Machine Translation, Translation

This project aims to identify the important aspects of translation quality of user reviews which will represent a starting point for developing better automatic MT metrics and challenge test sets, and will be also helpful for developing MT systems for this genre. We work on two types of reviews: Amazon products and IMDb movies, written in English and translated into two closely related target languages, Croatian and Serbian.
ELITR: European Live Translator
Authors: Ondřej Bojar, Dominik Macháček, Sangeet Sagar, Otakar Smrž, Jonáš Kratochvíl, Ebrahim Ansari, Dario Franceschini, Chiara Canton, Ivan Simonini, Thai-Son Nguyen, Felix Schneider, Sebastian Stücker, Alex Waibel, Barry Haddow, Rico Sennrich, Philip Williams
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Machine Translation, speech-recognition, Speech Recognition, Translation

ELITR (European Live Translator) project aims to create a speech translation system for simultaneous subtitling of conferences and online meetings targetting up to 43 languages. The technology is tested by the Supreme Audit Office of the Czech Republic and by alfaview®, a German online conferencing system. Other project goals are to advance document-level and multilingual machine translation, automatic speech recognition, and automatic minuting.
Progress of the PRINCIPLE Project: Promoting MT for Croatian, Icelandic, Irish and Norwegian
Authors: Andy Way, Petra Bago, Jane Dunne, Federico Gaspari, Andre Kåsen, Gauti Kristmannsson, Helen McHugh, Jon Arild Olsen, Dana Davis Sheridan, Páraic Sheridan, John Tinsley
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Machine Translation, Translation

This paper updates the progress made on the PRINCIPLE project, a 2-year action funded by the European Commission under the Connecting Europe Facility (CEF) programme. PRINCIPLE focuses on collecting high-quality language resources for Croatian, Icelandic, Irish and Norwegian, which have been identified as low-resource languages, especially for building effective machine translation (MT) systems. We report initial achievements of the project and ongoing activities aimed at promoting the uptake of neural MT for the low-resource languages of the project.
MTUOC: easy and free integration of NMT systems in professional translation environments
Authors: Antoni Oliver
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Machine Translation, NMT, Translation

In this paper the MTUOC project, aiming to provide an easy integration of neural and statistical machine translation systems, is presented. Almost all the required software to train and use neural and statistical MT systems are released under free licences. However, their use is not always easy and intuitive and medium-high specialized skills are required. MTUOC project provides simplified scripts for preprocessing and training MT systems, and a server and client for easy use of the trained systems. The server is compatible with popular CAT tools for a seamless integration. The project also distributes some free engines.
INMIGRA3: building a case for NGOs and NMT
Authors: Celia Rico, María Del Mar Sánchez Ramos, Antoni Oliver
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: NMT

INMIGRA3 is a three-year project that builds on the work of two previous initi-atives: INMIGRA2-CM and CRISIS-MT . Together, they address the specific needs of NGOs in multilingual settings with a particular interest in migratory contexts. Work on INMIGRA3 concentrates in the analysis of how best can be NMT put to use for the purposes of translating NGOs documentation.
The Multilingual Anonymisation Toolkit for Public Administrations (MAPA) Project
Authors: Ēriks Ajausks, Victoria Arranz, Laurent Bié, Aleix Cerdà-i-Cucó, Khalid Choukri, Montse Cuadros, Hans Degroote, Amando Estela, Thierry Etchegoyhen, Mercedes García-Martínez, Aitor García-Pablos, Manuel Herranz, Alejandro Kohan, Maite Melero, Mike Rosner, Roberts Rozis, Patrick Paroubek, Artūrs Vasiļevskis, Pierre Zweigenbaum
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: De-identification

We describe the MAPA project, funded under the Connecting Europe Facility programme, whose goal is the development of an open-source de-identification toolkit for all official European Union languages. It will be developed since January 2020 until December 2021.
APE-QUEST: an MT Quality Gate
Authors: Heidi Depraetere, Joachim Van den Bogaert, Sara Szoc, Tom Vanallemeersch
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Translation

The APE-QUEST project (2018--2020) sets up a quality gate and crowdsourcing workflow for the eTranslation system of EC’s Connecting Europe Facility to improve translation quality in specific domains. It packages these services as a translation portal for machine-to-machine and machine-to-human scenarios.
MICE: a middleware layer for MT
Authors: Joachim Van den Bogaert, Tom Vanallemeersch, Heidi Depraetere
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Domain Adaptation, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER)

The MICE project (2018-2020) will deliver a middleware layer for improving the output quality of the eTranslation system of EC’s Connecting Europe Facility through additional services, such as domain adaptation and named entity recognition. It will also deliver a user portal, allowing for human post-editing.
Neural Translation for the European Union (NTEU) Project
Authors: Laurent Bié, Aleix Cerdà-i-Cucó, Hans Degroote, Amando Estela, Mercedes García-Martínez, Manuel Herranz, Alejandro Kohan, Maite Melero, Tony O’Dowd, Sinéad O’Gorman, Mārcis Pinnis, Roberts Rozis, Riccardo Superbo, Artūrs Vasiļevskis
Date: 2020-11-01 00:00:00
Proceeding: EAMT 2020 11
Tasks: Translation

The Neural Translation for the European Union (NTEU) project aims to build a neural engine farm with all European official language combinations for eTranslation, without the necessity to use a high-resourced language as a pivot. NTEU started in September 2019 and will run until August 2021.