Smart Pre- and Post-Processing for STAR MT Translate (2018)

Abstract: After many successful experiments it has become evident that smart pre- and post-processing can significantly improve the output of neural machine translation. Therefore, various generic and language-specific processes are applied to the training corpus, the user input and the MT output for STAR MT Translate.

## star mt translate star mt translate is star's web-based mt system which can also be integrated in microsoft office products using the star mt office connector. in this mt application all data is kept safe within the customer's environment. the aim is to provide "useful" translations for customers in their company-specific domain, e.g. transportation. typically, the users are not translators but various professionals, e.g. mechanics, who need to understand the information that is only available in a foreign language. in professional translation projects mostly structured text is translated, possibly supported by mt. the user input to star mt translate however is much more flexible and "unpredictable". it contains customer-specific terminology but it consists of a wide range of linguistic phenomena, including ungrammatical sequences. to meet both kinds of text, firstly, the core of the engines are built from customer-specific indomain translations; secondly, to deal with the variety of language usage, the engines are complemented by out-of-domain data, and they use neural technology. ## smart pre-and post-processing good training data is the essential requisite to obtain good mt. it must cover the language phenomena that are likely to occur in the user input sent to the mt system for translation. even if the corpus includes the required characteristics it usually also contains "noise" that considerably reduces the quality of the mt output. therefore, star has developed a systematic strategy to identify and delete this "noise". firstly, language-independent processing rules delete incomplete formatting or punctuation, irrelevant characters, fragmentary sentences etc. nominalizations of various, inconsistent number formats (dates, decimal separators, etc.) as well as url and email addresses are defined in another step. a specific set of rules ensures that the corpora contain a balanced amount of similar sentences (regarding the length of sentences, the number of tokens, etc.) and determine the prioritization of segments depending on their completeness. finally, language-specific processes are applied that identify irrelevant text, e.g. typos, foreign language, informal expressions, anddepending on the languageinclude a special handling of morphological variants. the same generic processing steps and the adequate language-specific rules are applied to the input text, in order to send to the mt engine sentences that are made up in the same way as the ones it has learned. and finally, post-processing uses these rules, too, in order to obtain high-quality mt output. pérez-ortiz, sánchez-martínez, esplà-gomis, popović, rico, martins, van den bogaert, forcada (eds.) proceedings of the 21st annual conference of the european association for machine translation, p. 355 alacant, spain, may 2018.