A Template-based Method for Constrained Neural Machine Translation (2022)

Abstract: Machine translation systems are expected to cope with various types of constraints in many practical scenarios. While neural machine translation (NMT) has achieved strong performance in unconstrained cases, it is non-trivial to impose pre-specified constraints into the translation process of NMT models. Although many approaches have been proposed to address this issue, most existing methods can not satisfy the following three desiderata at the same time: (1) high translation quality, (2) high match accuracy, and (3) low latency. In this work, we propose a template-based method that can yield results with high translation quality and match accuracy and the inference speed of our method is comparable with unconstrained NMT models. Our basic idea is to rearrange the generation of constrained and unconstrained tokens through a template. Our method does not require any changes in the model architecture and the decoding algorithm. Experimental results show that the proposed template-based approach can outperform several representative baselines in both lexically and structurally constrained translation tasks.

## introduction constrained machine translation is of important value for a wide range of practical applications, such as interactive translation with user-specified lexical constraints @xcite @xcite , domain adaptation with in-domain dictionaries @xcite , and webpage translation with markup tags as structural constraints @xcite . developing constrained neural machine translation (nmt) approaches can make nmt models applicable to more real-world scenarios @xcite . however, it is challenging to directly impose constraints for nmt models due to their end-toend nature @xcite . in accordance with this problem, a branch of studies modifies the decoding algorithm to take the constraints into account when selecting candidates @xcite @xcite @xcite . although constrained decoding algorithms can guarantee the presence of constrained tokens, they can significantly slow down the translation process @xcite and can sometimes result in poor translation quality @xcite . another branch of works constructs synthetic data to help nmt models acquire the ability to translate with constraints @xcite @xcite . for instance, @xcite propose to inject markup tags into plain parallel texts to learn structurally constrained nmt models. the major drawback of data augmentation based methods is that they sometimes violate the constraints @xcite , limiting their application in constraint-critical situations. in this work, we use free tokens to denote the tokens that are not covered by the provided constraints. our motivation is to decompose the whole constrained translation task into the arrangement of constraints and the generation of free tokens. the constraints can be of many types, ranging from phrases in lexically constrained translation to markup tags in structurally constrained translation. intuitively, only arranging the provided constraints into the proper order is much easier than generating the whole sentence. therefore, we build a template by abstracting free token fragments into nonterminals, which are used to record the relative position of all the involved fragments. the template can be treated as a plan of the original sentence. the arrangement of constraints can be learned through a template generation sub-task. once the template is generated, we need some derivation rules to convert the nonterminals mentioned above into free tokens. each derivation rule shows the correspondence between a nonterminal and a free token fragment. these rules can be learned by the nmt model through semi-structured data. we call this sub-task template derivation. during inference, the model firstly generates the template and then extends each nonterminal in the template into natural language text. note that the two proposed sub-tasks can be accomplished through a single decoding pass. thus the decoding speed of our method is comparable with unconstrained nmt systems. by designing template format, our approach can cope with different types of constraints, such as lexical constraints, xml structural constraints, or markdown constraints. contributions in summary, the contributions of this work can be listed as follows: ## conclusion in this work, we propose a template-based framework for constrained translation and apply the framework to two specific tasks, which are lexically and structurally constrained translation. our motivation is to decompose the generation of the whole sequence into the arrangement of constraints and the generation of free tokens, which can be learned through a sequence-to-sequence framework. experiments demonstrate that the proposed method can achieve high translation quality and match accuracy simultaneously and our inference speed is comparable with unconstrained nmt baselines.