We present a joint probability model for statistical machine translation, which automatically learns word and phrase equivalents from bilingual corpora. Translations produced with parameters estimated using the joint model are more accurate than translations produced using IBM Model 4. Most of the noisy-channel-based models used in statistical machine translation (MT) (Brown et al., 1993) are conditional probability models. In the noisy-channel framework, each source sentence e in a parallel corpus is assumed to "generate" a target sentence f by means of a stochastic process, whose parameters are estimated using traditional EM techniques (Dempster et al., 1977) . The generative model explains how source words are mapped into target words and how target words are re-ordered to yield well-formed target sentences. A variety of methods are used to account for the re-ordering stage: word-based (Brown et al., 1993) , templatebased , and syntax-based , to name just a few. Although these models use different generative processes to explain how translated words are re-ordered in a target language, at the lexical level they are quite similar; all these models assume that source words are individually translated into target words. 1 1 The individual words may contain a non-existent element, called NULL. We suspect that MT researchers have so far chosen to automatically learn translation lexicons defined only over words for primarily pragmatic reasons. Large scale bilingual corpora with vocabularies in the range of hundreds of thousands yield very large translation lexicons. Tuning the probabilities associated with these large lexicons is a difficult enough task to deter one from trying to scale up to learning phrase-based lexicons. Unfortunately, trading space requirements and efficiency for explanatory power often yields non-intuitive results. Consider, for example, the parallel corpus of three sentence pairs shown in Figure 1 . Intuitively, if we allow any Source words to be aligned to any Target words, the best alignment that we can come up with is the one in Figure 1 .c. Sentence pair (S2, T2) offers strong evidence that "b c" in language S means the same thing as "x" in language T. On the basis of this evidence, we expect the system to also learn from sentence pair (S1, T1) that "a" in language S means the same thing as "y" in language T. Unfortunately, if one works with translation models that do not allow Target words to be aligned to more than one Source word -as it is the case in the IBM models (Brown et al., 1993) -it is impossible to learn that the phrase "b c" in language S means the same thing as word "x" in language T. The IBM Model 4 (Brown et al., 1993) , for example, converges to the word alignments shown in Figure 1 .b and learns the translation probabilities shown in Figure 1 .a. 2 Since in the IBM model one cannot link a Target word to more than a Source word, the training procedure 2 To train the IBM-4 model, we used Giza (Al-Onaizan et al., 1999) .
In REF , a joint probability model is presented which estimates phrase translation probabilities from a parallel corpus.
A Phrase-Based, Joint Probability Model For Statistical Machine Translation
Conference On Empirical Methods In Natural Language Processing
Computer Science