Bilingual word alignment forms the foundation of most approaches to statistical machine translation. Current word alignment methods are predominantly based on generative models. In this paper, we demonstrate a discriminative approach to training simple word alignment models that are comparable in accuracy to the more complex generative models normally used. These models have the the advantages that they are easy to add features to and they allow fast optimization of model parameters using small amounts of annotated data. Bilingual word alignment is the first step of most current approaches to statistical machine translation. Although the best performing systems are "phrasebased" (e.g, Och and Ney, 2004) , possible phrase translations are normally first extracted from wordaligned bilingual text segments. The standard approach to word alignment makes use of various combinations of five generative models developed at IBM by Brown et al. (1993) , sometimes augmented by an HMM-based model or Och and Ney's "Model 6" (Och and Ney, 2003) . The best combinations of these models can produce high accuracy alignments, at least when trained on a large corpus of fairly direct translations in related languages. These standard models are less than ideal, however, in a number of ways, two of which we address in this paper. First, although the standard models can theoretically be trained without supervision, in practice various parameters are introduced that should be optimized using annotated data. For, example, Och and Ney (2003) suggest supervised optimization of a number of parameters, including the probablity of jumping to the empty word in the HMM model, as well as smoothing parameters for the distortion probabilities and fertility probabilities of the more complex models. Since the values of these parameters affect the values of the translation, alignment, and fertility probabilities trained by EM, there is no effective way to optimize them other than to run the training procedure with a particular combination of values and evaluate the accuracy of the resulting alignments. Since evaluating each combination of parameter values in this way can take hours to days on a large training corpus, it seems safe to say that these parameters are rarely if ever truly jointly optimized for a particular alignment task. The second problem we address is the difficulty of adding features to the standard generative models. Generative models require a generative "story" as to how the observed data is generated by an interrelated set of stochastic processes. For example, the generative story for IBM Models 1 and 2 and the HMM alignment model is that a target language translation of a given source language sentence is generated by first choosing a length for the target language sentence, then for each target sentence position choosing a source sentence word, and then choosing the corresponding target language word. When Brown et al. (1993) wanted to add a fertility component to create Models 3, 4, and 5, however, this generative 81
What is more, it is difficult to add characteristics to the standard generative models REF .
A Discriminative Framework For Bilingual Word Alignment
Human Language Technology Conference And Empirical Methods In Natural Language Processing
Computer Science