How Should Markup Tags Be Translated? (2020)

Abstract: The ability of machine translation (MT) models to correctly place markup is crucial to generating high-quality translations of formatted input. This paper compares two commonly used methods of representing markup tags and tests the ability of MT models to learn tag placement via training data augmentation. We study the interactions of tag representation, data augmentation size, tag complexity, and language pair to show the drawbacks and benefits of each method. We construct and release new test sets containing tagged data for three language pairs of varying difficulty.

## introduction the quality of machine translation (mt) has drastically improved in recent years, making mt technology more widely used than ever before in applications ranging from financial services @xcite to fashion, social media, and other user-generated content @xcite @xcite , among other tasks. a large amount of content requiring translation is not isolated plain text. rather, it originates in the context of structured documents, using document format specifications such as html, microsoft word, pdf, etc. currently, the translation industry addresses the translation of structured documents by dividing the task between a translation management system (tms) and an underlying mt system (e.g. @xcite ). figure 1 shows a schematic of the process. the tms parses, manipulates, and validates the higher-level document structure. it is responsible for finding the translatable portions of the input document, performing sentence segmentation on the content, sending it to an underlying mt system for translation, and placing the result back into the document structure. for example, the tms may pass over material contained in @xmath0...</@xmath1, while sending to mt the contents @xmath2...</@xmath3..."/> attributes. properly transferring formatting tags within the translatable content (bold, italic, hyperlink, superscript, etc.) remains the responsibility of the mt process rather than the tms. the correct preservation and transfer of inline markup from source to target thus forms a crucial component of the overall quality of the mt system. an accurate placement within the segment of inline markup provides a more readable and usable document in its raw mt form, in addition to saving human time and effort if post-editing is used. despite the key role of the mt system in processing structured documents -both in terms of tms expectations and in lowering translation costs -the setup of translation in the context of structure is rarely addressed in the standard mt evaluation benchmarks. they typically focus on the pure-content, string-based tasks @xcite . in this paper, we study the question of how inline markup should be represented in and processed by the mt system in order to result in the highest placement accuracy. given the deep semantic representations and generalization abilities provided by modern neural mt systems, we design a series of experiments to test their capabilities specifically on the problem of markup transfer within the translation process. the paper makes the following contributions: ## related work various works have addressed the inline markup problem in the context of statistical mt systems. @xcite extensively summarize many such methods. their "two-stream" approach is quite similar to our detag-and-project baseline, relying on phrase and word alignments and carefully designed tag transfer rules to re-insert inline markup into the translated content. though the accuracy of this approach is tested in a small human evaluation, it is explicitly not compared to any other method of tag representation. @xcite addresses the same question, comparing five different varieties of detag-and-project and mask-based approaches. the work concludes that the detag-and-project approach of @xcite performs the best in complicated tagging scenarios, while masking can be a strong approach in simplistic cases. moving to neural mt, @xcite experiment with "raw" tags that are left unmodified in the input. the authors test a constrained beam search that restricts the decoder to outputting all and only the tags actually present in the source while maintaining xml well-formedness. they also introduce a pointer mechanism @xcite to promote copy-through of tags and other non-translatable content. these restrictions are shown to improve output, but no other tag representation aside from raw tags is tested. we thus position our work as spanning and unifying elements of these previous studies, directly comparing the major tag representation methods with a minimum of other modeling changes in a state-of-the-art neural mt setting. additionally we introduce a data augmentation technique that removes the dependence on pre-existing tagged training data assumed by the approaches proposed by e.g. @xcite . in this context, we also note complementary work on the implicit learning of structure by sequence-to-sequence models. target-side struc-ture expressed as content, without special annotations, has been used in string-to-tree syntax-based mt by @xcite and @xcite ; the latter group notes that well-formed parse trees are generated 99.5% of the time. the masking method of passing abstracted information between input and output has been applied in mt to non-translatables such as numbers, urls, named entities, etc. @xcite . it has been shown to improve automatic metric scores, with the masks correctly appearing in the mt output a very high percentage of the time @xcite . source factors, or additional input streams provided in lockstep with the content to translate @xcite , are one mechanism for signaling structural information in the input while leaving it accessible to the model. it has been used to augment an mt system with terminology constraints @xcite , and the same mechanism could be in principle applied to tags as well. training data augmentation is a popular way to increase model learning and robustness for specific phenomena. of particular interest is the finding by @xcite showing that a "balanced diet" of training data synthetically augmented with spelling errors improved model performance on natural errors, without harming performance on clean text. our notion regarding synthetically injected and naturally occurring tags is similar. ## data augmentation via tag injection the vast majority of data available for mt research appears as plain text only. 1 while tagged training corpora, when available, will inevitably be limited in size, domains covered, and language availability, we propose as an alternative a general technique for injecting markup tags of controlled complexity into any parallel corpus with high accuracy. although not always adhering to the strict syntax of a well-defined markup language, most inline tags follow the xml standard and create a hierarchical structure within a segment by introducing either paired tags (opening and closing) or selfclosing unary tags. this structure is expected to be preserved by the translation, with paired tags surrounding corresponding text fragments. in figure 1 , the italicized fragment improved transfers to ital-icized améliorée. we propose a method to identify such corresponding fragments in the source and target sides of parallel data and to automatically inject tags based on them. ## conclusion we have performed a comprehensive evaluation of several tag representation methods and proposed a data-augmentation technique that allows mt models to jointly learn content translation and inline tag placement. results show that representing tags as masks, together with data augmentation, leads to equivalent or improved performance over a detag-and-project approach: placement accuracy is higher for tags frequent in the training data, while it may vary for tags never observed in context. in practice, it may be preferable to rely on the mt model's ability to learn mask placement -even with some variability in accuracy -than to implement, debug, and maintain the baseline's more complicated projection rules and the required alignment model. raw tags, on the other hand, fail our generalization tests. though placement accuracy is again baseline-beating for commonly observed tags, raw models seem unable to copy rare tags into the output without a significant number of mutilations, deletions, and duplications: an unacceptable result for the goal of obtaining well-structured output. several changes to our setup may improve the transfer of raw tags. injecting xliff tags with a wider variety of id values is needed to expose the model to them in context instead of merely in self-translation. explicitly identifying tag tokens via input factors @xcite , or constraining/promoting the output of complete tags @xcite , would also be helpful for reducing the rate of malformed output.