Joint Translation and Unit Conversion for End-to-end Localization (2020)

Abstract: A variety of natural language tasks require processing of textual data which contains a mix of natural language and formal languages such as mathematical expressions. In this paper, we take unit conversions as an example and propose a data augmentation technique which lead to models learning both translation and conversion tasks as well as how to adequately switch between them for end-to-end localization.

## introduction neural networks trained on large amounts of data have been shown to achieve state-of-the art solutions on most nlp tasks such as textual entailment, question answering, translation, etc. in particular, these solutions show that one can successfully model the ambiguity of language by making very few assumptions about its structure and by avoiding any formalization of language. however, unambiguous, formal languages such as numbers, mathematical expressions or even programming languages (e.g. markup) are abundant in text and require the ability to model the symbolic, "procedural" behaviour governing them. @xcite . an example of an application where such examples are frequent is the extension of machine translation to localization. localization is the task of combining translation with "culture adaptation", which involves, for instance, adapting dates @xcite , calendar conversions (march 30, 2019 to rajab 23, 1441 in hijri calendar) or conversions of currencies or of units of measure (10 kgs to 22 pounds). current approaches in machine translation handle the processing of such sub-languages in one of two ways: the sub-language does not receive any special treatment but it may be learned jointly with the main task if it is represented enough in the data. alternatively, the sub-language is decoupled from the natural text through pre/post processing techniques: e.g. a miles expression is converted into kilometers in a separate step after translation. arguably the first approach can successfully deal with some of these phenomena: e.g. a neural network may learn to invoke a simple conversion rule for dates, if enough examples are seen training. however, at the other end of the spectrum, correctly converting distance units, which itself is a simple algorithm, requires knowledge of numbers, basic arithmetic and the specific conversion function to apply. it is unrealistic to assume a model could learn such conversions from limited amounts of parallel running text alone. furthermore, this is an unrealistic task even for distributional, unsupervised pre-training (turney and pantel, 2010; @xcite , despite the success of such methods in capturing other nonlinguistic phenomena such as world knowledge or cultural biases @xcite ). 1 while the second approach is currently the preferred one in translation technology, such decoupling methods do not bring us closer to end-to-end solutions and they ignore the often tight interplay of the two types of language: taking unit conversion as an example, approximately 500 miles, should be translated into ungefähr 800 km (approx. 800km) and not ungefähr 804 km (approx. 804km). in this paper we highlight several of such language mixing phenomena related to the task of localization for translation and focus on two distance (miles to kilometers) and temperature (fahrenheit to celsius) conversion tasks. specifically, we per-form experiments using the popular mt transformer architecture and show that the model is successful at learning these functions from symbolically represented examples. furthermore, we show that data augmentation techniques together with small changes in the input representation produce models which can both translate and appropriately convert units of measure in context. ## related work several theoretical and empirical works have addressed the computational capabilities end expressiveness of deep learning models. theoretical studies on language modeling have mostly targeted simple grammars from the chomsky hierarchy. in particular, @xcite proves that transformer networks suffer limitations in modeling regular periodic languages (such as a n b n ) as well as hierarchical (context-free) structures, unless their depth or self-attention heads increase with the input length. on the other hand, @xcite proves that lstm networks can recognize a subset of periodic languages. also experimental papers analyzed the capability of lstms to recognize these two language classes @xcite @xcite @xcite , as well as natural language hierarchical structures @xcite . it is worth noticing, however, that differently from formal language recognition tasks, state of the art machine translation systems @xcite are still based on the transformer architecture . other related work addresses specialized neural architectures capable to process and reason with numerical expressions for binary addition, evaluating arithmetic expressions or other number manipulation tasks @xcite @xcite . while this line of work is very relevant, we focus on the natural intersection of formal and everyday language. the types of generalization that these studies address, such as testing with numbers orders of magnitude larger than those in seen in training, are less relevant to our task. the task of solving verbal math problems @xcite @xcite specifically addresses natural language mixed with formal language. similarly, @xcite introduces a benchmark for evaluating quantitative reasoning in natural language inference and @xcite one for symbolic operations such as addition or sorting in reading comprehension. however these papers show the best results with twostep approaches, which extract the mathematical or symbolic information from the text and further manipulate it analytically. we are not aware of any other work successfully addressing both machine translation and mathematical problems, or any of the benchmarks above, in an end-to-end fashion. ## unit conversion in mt localization the goal of localization is to enhance plain content translation so that the final result looks and feels as being created for a specific target audience. parallel corpora in general include localization of formats numeric expressions (e.g. from 1,000,000.00 (en-us) to 1.000.000,00 (de-de)). format conversions in most of the cases reduce to operations such as reordering of elements and replacement of symbols, which quite naturally fit inside the general task of machine translation. in this paper, we are interested in evaluating the capability of neural mt models to learn less natural operations, which are typically involved in the conversion of time expressions (e.g. 3:30pm → 15:30) and units of measure, such as lengths (10ft to 3m) and temperatures (55f to 12.8c). we choose two measure unit conversion tasks that are very prevalent in localization: fahrenheit to celsius temperature conversion and miles to kilometers. we address the following questions: 1) can a standard nmt architecture, the transformer, be used to learn the functions associated with these two conversion tasks (section 3.1) and 2) can the same architecture be used to train a model that can do both mt and unit conversion? (section 3.2) ## conclusions we have outlined natural/formal language mixing phenomena in the context of end-to-end localization for mt and have proposed a data augmentation method for learning unit conversions in context. surprisingly, the results show not only that a single architecture can learn both translation and unit conversions, but can also appropriately switch between them when a small amount of localization data is used in training. for future work we plan to create a diverse localization test suite and investigate if implicit learning of low-level concepts such as natural numbers takes place and if unsupervised pre-training facilitates such learning.