Comparison of two different interactive translation prediction approaches integrated in a computer-assisted translation workbench

The objective of interactive translation prediction (ITP), a variant of computer-assisted translation (CAT), is to help professional translators by offering them context-based computer-generated suggestions as they type; the professional translators evaluate then the suggestions, and either accept one of them (sometimes, a prefix can be selected instead of the full proposal) or ignore them all and keep typing the translation in their own way. Most state-of-the-art ITP approaches, such as the one described by Barrachina et al. (2009) or the Thot system (Ortiz-Martı́nez and Casacuberta, 2014) obtain the suggestions by means of a modified (or tailor-made) statistical machine translations system (SMT) (Koehn, 2004) that is able to provide additional information (such as word alignments, alternative translations, and scores or probabilities for the translation); the suggestions range from a few words to complete translations of the sentence constrained by the already typed target-language prefix. These ITP systems are able to leverage more information from the bilingual resource than if it was used unmodified as a black-box, but doing so they inherit the common requirements of SMT, namely, the dependency on the availability of extensive parallel corpora. It is worth noting that integrating other resources of bilingual information would be almost impossible in this kind of systems, as the ITP tool needs the additional information obtained from the underlying SMT engine. The authors of this presentation have already proposed an ITP approach (Pérez-Ortiz et al., 2014) that, unlike those previously described, is able to use any bilingual resource capable of delivering one or more translations into target language, regardless of how they are obtained and without the need to modify the resource; suggestions are created by generating all possible sub-segments of words in the source-language sentence (up to a given length) and then querying the available bilingual resources for their translations. The nature of these bilingual resources is not limited to MT systems, but they may also include translation memories, dictionaries, catalogues of bilingual phrases, or any combination of them. A neural-based machine learning algorithm trained on features extracted from the source sentence, on the current prefix of the target sentence, and on the translated sub-segments is used to rank and select which suggestions to show at each time step. Not having to rely on the inner workings of each system allows us to 1Thot can be used both as a statistical machine translation (SMT) system and as an ITP tool. We will refer to each system as SMT Thot and ITP Thot, respectively. 2A paper describing this neural system will be presented at the main AMTA 2016 conference. This presentation, however, will focus on an original comparison between our approach and Thot. seamlessly integrate new resources without modifying how the ITP system works. In this presentation, we will compare for the first time our proposal with the state-of-the-art system Thot (Ortiz-Martı́nez and Casacuberta, 2014). ITP Thot obtains the proposals as complete translations of the sentences constrained by the typed prefix, and allows the user to accept a prefix or the whole of them. In order to allow a fair comparison of the systems, we will use SMT Thot as a black-box bilingual resource for our approach. We will present the results of a comparison between our approach and Thot, both with an extensive automatic evaluation and with a preliminary qualitative and quantitative human evaluation. For the preliminary human evaluation, we will evaluate the usage of both ITP approaches integrated into the widely used open-source professional CAT tool OmegaT. OmegaT has a robust plug-in mechanism that allows us to integrate both approaches as part of the native interface already known by many professional translators; another plug-in is used to measure and log user performance transparently.