UvA-DARE ( Digital Academic Repository ) Latent Domain Translation Models in Mix-of-Domains Haystack

This paper addresses the problem of selecting adequate training sentence pairs from a mix-ofdomains parallel corpus for a translation task represented by a small in-domain parallel corpus. We propose a novel latent domain translation model which includes domain priors, domaindependent translation models and language models. The goal of learning is to estimate the probability of a sentence pair in mix-domain corpus to be inor out-domain using in-domain corpus statistics as prior. We derive an EM training algorithm and provide solutions for estimating out-domain models (given only inand mix-domain data). We report on experiments in data selection (intrinsic) and machine translation (extrinsic) on a large parallel corpus consisting of a mix of a rather diverse set of domains. Our results show that our latent domain invitation approach outperforms the existing baselines significantly. We also provide analysis of the merits of our approach relative to existing approaches. Large parallel corpora are important for training statistical MT systems. Besides size, the relevance of a parallel training corpus to the translation task at hand can be decisive for system performance, cf. (Axelrod et al., 2011; Koehn and Haddow, 2012). In this paper we look at data selection where we have access to a large parallel data repository Cmix, representing a rather varied mix of domains, and we are given a sample of in-domain parallel data Cin, exemplifying a target translation task. Simply concatenating Cin with Cmix does not always deliver best performance, because including irrelevant sentences might be more harmful than beneficial, cf. (Axelrod et al., 2011). To make the best of available data, we must select sentences from Cmix for their relevance to translating sentences from Cin. Axelrod et al. (2011) and follow-up work, e.g., (Haddow and Koehn, 2012; Koehn and Haddow, 2012), select sentence pairs in Cmix using the cross-entropy difference between inand mix-domain language models, both source and target sides, a modification of the Moore and Lewis method (Moore and Lewis, 2010). In the translation context, however, often a source phrase has different senses/translations in different domains, which cannot be distinguished with monolingual language models. The dependence of translation choice on domain suggests that the word alignments themselves can better be conditioned on domain information. However, in the data selection setting, corpus Cmix often does not contain useful domain markers, and Cin contains only a small sample of in-domain sentence pairs. In this paper we present a latent domain translation model which weights every sentence pair 〈f , e〉 ∈ Cmix with a probability P (D | f , e) for being in-domain (D1) or out-domain (D0). Our model defines P (e, f) = ∑ D∈{D1,D0} P (D)P (e, f | D), using a latent domain variable D ∈ {D0, D1}. Using bidirectional translation models, this leads to a domain prior P (D), domain-dependent translation models Pt(· |·, D) and language models Plm(· | D) as in Equation 1: P (e, f | D) = 1 2 × {Plm(e | D)Pt(f | e, D) + Plm(f | D)Pt(e | f , D)} (1) This work is licenced under a Creative Commons Attribution 4.0 International License. Page numbers and proceedings footer are added by the organizers. License details: http://creativecommons.org/licenses/by/4.0/