A Comparison of Pivot Methods for Phrase-based Statistical MachineTranslation
Masao Utiyama and Hitoshi Isahara
National Institute of Information and Communications Technology
3-5 Hikari-dai, Soraku-gun, Kyoto 619-0289 Japan{mutiyama,isahara}@nict.go.jp

Abstract
We compare two pivot strategies forphrase-based statistical machine translation (SMT), namely phrase translationand sentence translation. The phrasetranslation strategy means that we directly construct a phrase translation table (phrase-table) of the source and target language pair from two phrase-tables;
one constructed from the source language
and English and one constructed from English and the target language. We then usethat phrase-table in a phrase-based SMT
system. The sentence translation strategy means that we first translate a source
language sentence into n English sentences and then translate these n sentences
into target language sentences separately.
Then, we select the highest scoring sentence from these target sentences. We conducted controlled experiments using theEuroparl corpus to evaluate the performance of these pivot strategies as compared to directly trained SMT systems.
The phrase translation strategy significantly outperformed the sentence translation strategy. Its relative performance was
0.92 to 0.97 compared to directly trained
SMT systems.

1

Introduction

The rapid and steady progress in corpus-based machine translation (Nagao, 1981; Brown et al., 1993)

has been supported by large parallel corpora suchas the Arabic-English and Chinese-English parallel corpora distributed by the Linguistic Data Consortium and the Europarl corpus (Koehn, 2005),
which consists of 11 European languages. However, large parallel corpora do not exist for many
language pairs. For example, there are no publicly available Arabic-Chinese large-scale parallel
corpora even though there are Arabic-English andChinese-English parallel corpora.
Much work has been done to overcome the lackof parallel corpora. For example, Resnik and Smith(2003) propose mining the web to collect parallel
corpora for low-density language pairs. Utiyama
and Isahara (2003) extract Japanese-English parallel
sentences from a noisy-parallel corpus. Munteanu
and Marcu (2005) extract parallel sentences fromlarge Chinese, Arabic, and English non-parallelnewspaper corpora.
Researchers can also make the best use of existing (small) parallel corpora. For example, Nießen
and Ney (2004) use morpho-syntactic information totake into account the interdependencies of inflected
forms of the same lemma in order to reduce theamount of bilingual data necessary to sufficiently
cover the vocabulary in translation. Callison-Burchet al. (2006a) use paraphrases to deal with unknown
source language phrases to improve coverage andtranslation quality.
In this paper, we focus on situations where no parallel corpus is available (except a few hundred parallel sentences for tuning parameters). To tackle these
extremely scarce training data situations, we propose using a pivot language (English) to bridge the484
Proceedings of NAACL HLT 2007, pages 484–491,
Rochester, NY, April 2007. c 2007 Association for Computational Linguistics

source and target languages in translation. We firsttranslate source language sentences or phrases into
English and then translate those English sentencesor phrases into the target language, as described in
Section 3. We thus assume that there is a parallel
corpus consisting of the source language and English as well as one consisting of English and the target language. Selecting English as a pivot language
is a reasonable pragmatic choice because English isincluded in parallel corpora more often than otherlanguages are, though any language can be used as apivot language.
In Section 2, we describe a phrase-based statistical machine translation (SMT) system that was used
to develop the pivot methods described in Section
3. This is the shared task baseline system for the2006 NAACL/HLT workshop on statistical machinetranslation (Koehn and Monz, 2006) and consists of
the Pharaoh decoder (Koehn, 2004), SRILM (Stolcke, 2002), GIZA++ (Och and Ney, 2003), mkcls(Och, 1999), Carmel,1 and a phrase model training
code.

2

Phrase-based SMT

We use a phrase-based SMT system, Pharaoh,(Koehn et al., 2003; Koehn, 2004), which is basedon a log-linear formulation (Och and Ney, 2002). It
is a state-of-the-art SMT system with freely available software, as described in the introduction.
The system segments the source sentence into socalled phrases (a number of sequences of consecutive words). Each phrase is translated into a target
language phrase. Phrases may be reordered.
Let f be a source sentence (e.g, French) and e be atarget sentence (e.g., English), the SMT system outputs an ê that satisfiesê = arg max Pr(e|f )e= arg maxeMX

λm hm (e, f )

(1)
(2)

m=1

where hm (e, f ) is a feature function and λm is aweight. The system uses a total of eight featurefunctions: a trigram language model probability ofthe target language, two phrase translation probabilities (both directions), two lexical translation prob1

http://www.isi.edu/licensed-sw/carmel/

485

abilities (both directions), a word penalty, a phrase
penalty, and a linear reordering penalty. For details
on these feature functions, please refer to (Koehn etal., 2003; Koehn, 2004; Koehn et al., 2005). To set
the weights, λm , we carried out minimum error rate
training (Och, 2003) using BLEU (Papineni et al.,2002) as the objective function.

3 Pivot methods
We use the phrase-based SMT system described in
the previous section to develop pivot methods. We
use English e as the pivot language. We use Frenchf and German g as examples of the source and target
languages in this section.
We describe two types of pivot strategies, namely
phrase translation and sentence translation.
The phrase translation strategy means that we directly construct a French-German phrase translationtable (phrase-table for short) from a French-English
phrase-table and an English-German phrase-table.
We assume that these French-English and EnglishGerman tables are built using the phrase model training code in the baseline system described in the
introduction. That is, phrases are heuristically extracted from word-level alignments produced by doing GIZA++ training on the corresponding parallel
corpora (Koehn et al., 2003).
The sentence translation strategy means that wefirst translate a French sentence into n English sentences and translate these n sentences into German
separately. Then, we select the highest scoring sentence from the German sentences.
3.1

Phrase translation strategy

The phrase translation strategy is based on the factthat the phrase-based SMT system needs a phrasetable and a language model for translation. Usually,
we have the language model of a target language.
Consequently, we only need to construct a phrasetable to train the phrase-based SMT system.
We assume that we have a French-English phrasetable TF E and an English-German phrase-table
TEG . From these tables, we construct a FrenchGerman phrase-table TF G , which requires estimating four feature functions; phrase translation probabilities for both directions, φ(f¯|ḡ) and φ(ḡ|f¯) andlexical translation probabilities for both directions,pw (f¯|ḡ) and pw (ḡ|f¯), where f¯ and ḡ are French andGerman phrases that are parts of phrase translation
pairs in TF E and TEG , respectively.2
We estimate these probabilities using the probabilities available in TF E and TEG as follows.3
φ(f¯|ḡ) =X

φ(f¯|ē)φ(ē|ḡ)(3)

φ(ḡ|ē)φ(ē|f¯)(4)ē∈TF E ∩TEG

φ(ḡ|f¯) =Xē∈TF E ∩TEG

pw (f¯|ḡ) =X

pw (f¯|ē)pw (ē|ḡ) (5)ē∈TF E ∩TEG

pw (ḡ|f¯) =X

pw (ḡ|ē)pw (ē|f¯) (6)ē∈TF E ∩TEG

where ē ∈ TF E ∩TEG means that the English phraseē is included in both TF E and TEG as part of phrasetranslation pairs. φ(f¯|ē) and φ(ē|f¯) are phrasetranslation probabilities for TF E and φ(ē|ḡ) andφ(ḡ|ē) are those for TEG . pw (f¯|ē) and pw (ē|f¯) arelexical translation probabilities for TF E and pw (ē|ḡ)
and pw (ḡ|ē) are those for TEG .
The definitions of the phrase and lexical translation probabilities are as follows (Koehn et al., 2003).
count(f¯, ē)¯0f¯0 count(f , ē)φ(f¯|ē) = P(7)

where count(f¯, ē) gives the total number of times
the phrase f¯ is aligned with the phrase ē in the parallel corpus. Eq. 7 means that φ(f¯|ē) is calculatedusing maximum likelihood estimation.
The definition of the lexical translation probability is
pw (f¯|ē) = max pw (f¯|ē, a)(8)
pw (f¯|ē, a) =anYEw (fi |ē, a)

(9)

i=1

Ew (fi |ē, a) =X
1
w(fi |ej )|{j|(i, j) ∈ a}| ∀(i,j)∈a(10)2

Feature functions scores are calculated using these probabilities. For example, for a translation probability of a Frenchsentence f = f¯1 .Q. . f¯K and a German sentence g = ḡ1 . . . ḡK ,Kh(g, f ) = log i=1 φ(f¯i |ḡi ), where K is the number ofphrases.
3
Wang et al. (2006) use essentially the same definition to
induce the translation probability of the source and target language word alignment that is bridged by an intermediate language. Callison-Burch et al. (2006a) use a similar definition for
a paraphrase probability.

486

count(f, e)0
f 0 count(f , e)w(f |e) = P(11)

where count(f, e) gives the total number of times
the word f is aligned with the word e in the parallel corpus. Thus, w(f |e) is the maximum likelihood estimation of the word translation probabilityof f given e. Ew(fi |ē, a) is calculated from a wordalignment a between a phrase pair f¯ = f1 f2 . . . fn
and ē = e1 e2 . . . em where fi is connected to several(|{j|(i, j) ∈ a}|) English words. Thus, Ew(fi |ē, a)is the average (or mixture) of w(fi |ej ). This meansthat Ew(fi |ē, a) is an estimation of the probability of fi in a. Consequently, pw (f¯|ē, a) estimates
the probability of f¯ given ē and a using the product of the probabilities Ew(fi |ē, a). This assumes
that the probability of fi is independent given ē anda. pw (f¯|ē) takes the highest pw (f¯|ē, a) if there
are multiple alignments a. This discussion, whichis partly based on Section 4.1.2 of (Och and Ney,
2004), means that the lexical translation probability
pw (f¯|ē) is another probability estimated using theword translation probability w(f |e).
The justification of Eqs. 3–6 is straightforward.
From the discussion above, we know that the probabilities, φ(f¯|ē), φ(ē|f¯), φ(ḡ|ē), φ(ē|ḡ), pw (f¯|ē),pw (ē|f¯), pw (ḡ|ē), and pw (ē|ḡ) are probabilities in
the ordinary sense. Thus, we can derive φ(f¯|ḡ),φ(ḡ|f¯), pw (f¯|ḡ), and pw (ḡ|f¯) by assuming thatthese probabilities are independent given an English
phrase ē (e.g., φ(f¯|ḡ, ē) = φ(f¯|ē)).
We construct a TF G that consists of all FrenchGerman phrases whose phrase and lexical translation probabilities as defined in Eqs. 3–6 are greaterthan 0. We use the term PhraseTrans to denote SMT
systems that use the phrase translation strategy described above.
3.2

Sentence translation strategy

The sentence translation strategy uses two independently trained SMT systems. We first translate a French sentence f into n English sentences
e1 , e2 , ..., en using a French-English SMT system.
Each ei (i = 1 . . . n) has the eight scores calculated from the eight feature functions described in
Section 2. We denote these scores hei1 , hei2 , . . . hei8 .
Second, we translate each ei into n German sentences gi1 , gi2 , . . . , gin using an English-German

SMT system. Each gij (j = 1 . . . n) has the eightscores, which are denoted as hgij1 , hgij2 , . . . , hgij8 .
This situation is depicted as
f

→ ei
→ gij

(hei1 , hei2 , . . . , hei8 )(hgij1 , hgij2 , . . . , hgij8 )We define the score of gij , S(gij ), as
8
X

S(gij ) =(λem heim + λgm hgijm )(12)types of parallel corpora; the source language–
English, English–the target language, and the source
language–the target language. This means that we
can directly train an SMT system using the source
and target language parallel corpus as well as pivot
SMT systems using English as the pivot language.
We use the term Direct to denote directly trained
SMT systems. For each language pair, we compare four SMT systems; Direct, PhraseTrans, SntTrans15, and SntTrans1.5

m=1where λem and λgm are weights set by performing
minimum error rate training4 as described in Section
2. We select the highest scoring German sentenceĝ = arg max S(gij )
gij

(13)

as the translation of the French sentence f .
A drawback of this strategy is that translation
speed is about O(n) times slower than those of thecomponent SMT systems. This is because we haveto run the English-German SMT system n times fora French sentence. Consequently, we cannot set nvery high. When we used n = 15 in the experiments described in Section 4, it took more than twodays to translate 3064 test sentences on a 3.06GHz
LINUX machine.
Note that when n = 1, the above strategy produces the same translation with the simple sequential method that we first translate a French sentence
into an English sentence and then translate that sentence into a German sentence.
We use the terms SntTrans15 and SntTrans1 to denote SMT systems that use the sentence translation
strategy with n = 15 and n = 1, respectively.

4

Experiments

We conducted controlled experiments using theEuroparl corpus. For each language pair described below, the Europarl corpus provides three4

We use a reranking strategy for the sentence translation
strategy. We first obtain n2 German sentences for each Frenchsentence by applying two independently trained French-English
and English-German SMT systems. Each of the translated German sentences has the sixteen scores as described above. The
weights in Eq. 12 are tuned against reference German sentencesby performing minimum error rate training. These weights arein general different from those of the original French-English
and English-German SMT systems.

487

4.1

Training, tuning and testing SMT systems

We used the training data for the shared task of
the SMT workshop (Koehn and Monz, 2006) totrain our SMT systems. It consists of three parallel corpora: French-English, Spanish-English, andGerman-English.
We used these three corpora to extract a set ofsentences that were aligned to each other across all
four languages. For that purpose, we used English
as the pivot. For each distinct English sentence, we
extracted the corresponding French, Spanish, andGerman sentences. When an English sentence occurred multiple times, we extracted the most frequent translation. For example, because “Resumption of the session” was translated into “Wiederaufnahme der Sitzungsperiode” 120 times and “Wiederaufnahme der Sitzung” once, we extracted “Wiederaufnahme der Sitzungsperiode” as its translation.
Consequently, we extracted 585,830 sentences foreach language. From these corpora, we constructed
the training parallel corpora for all language pairs.
We followed the instruction of the shared taskbaseline system to train our SMT systems.6 We
used the trigram language models provided with theshared task. We did minimum error rate training on
the first 500 sentences in the shared task development data to tune our SMT systems and used the5
As discussed in the introduction, we intend to use the pivot
strategies in a situation where a very limited amount of paralleltext is available. The use of the Europarl corpus is not an accurate simulation of the intended situation because it enables us to
use a relatively large parallel corpus for direct training. However, it is necessary to evaluate the performance of the pivot
strategies against that of Direct SMT systems under controlled
experiments in order to determine how much the pivot strategies can be improved. This is a first step toward the use of pivot
methods in situations where training data is extremely scarce.
6
The parameters for the Pharaoh decoder were “-dl 4 -b 0.03
-s 100”. The maximum phrase length was 7.

3064 test sentences for each language as our test set.
Our evaluation metric was %BLEU scores, as calculated by the script provided along with the sharedtask.7 We lowercased the training, development andtest sentences.
4.2 Results
Table 1 compares the BLEU scores of the four SMT
systems; Direct, PhraseTrans, SntTrans15, and SntTrans1 for each language pair. The columns SE andET list the BLEU scores of the Direct SMT systems trained on the source language–English andEnglish–the target language parallel corpora. The
numbers in the parentheses are the relative scoresof the pivot SMT systems, which were obtainedby dividing their BLEU scores by that of the corresponding Direct system. For example, for the
Spanish–French language pair, the BLEU score of
the Direct SMT system was 35.78, that of thePhraseTrans SMT system was 32.90, and the relative performance was 0.92 = (32.90/35.78). For
the SntTrans15 SMT system, the BLEU score was
29.49 and the relative performance was 0.82 =(29.49/35.78).
The BLEU scores of the Direct SMT systems
were higher than those of the PhraseTrans SMT systems for all six source-target language pairs. The
PhraseTrans SMT systems performed better thanthe SntTrans15 SMT systems for all pairs. The
SntTrans15 SMT systems were better than the SntTrans1 SMT systems for four pairs. According
to the sign test, under the null hypothesis that the
BLEU scores of two systems are equivalent, finding
one system obtaining better BLEU scores on all six
language pairs is statistically significant at the 5 %level. Obtaining four better scores is not statisticallysignificant. Thus, Table 1 indicates
Direct > PhraseTrans > SntTrans15 ∼ SntTrans1
where “>” and “∼” means that the differences of
the BLEU scores of the corresponding SMT systems
are statistically significant and insignificant, respectively.
7

Callison-Burch et al. (2006b) show that in general a higher
BLEU score is not necessarily indicative of better translation
quality. However, they also suggest that the use of BLEU isappropriate for comparing systems that use similar translation
strategies, which is the case with our experiments.

488

As expected, the Direct SMT systems outperformed the other systems. We regard the BLEU
scores of the Direct systems as the upperbound. The
SntTrans15 SMT systems did not significantly outperform the SntTrans1 SMT systems. We think that
this is because n = 15 was not large enough to covergood translation candidates.8 Selecting the highest
scoring translation from a small pool did not always
lead to better performance. To improve the performance of the sentence translation strategy, we need
to use a large n. However, this is not practical because of the slow translation speed, as discussed in
Section 3.2.
The PhraseTrans SMT systems significantly outperformed the SntTrans15 and SntTrans1 systems.
That is, the phrase translation strategy is better
than the sentence translation strategy. Since thephrase-tables constructed using the phrase translation strategy can be integrated into the Pharaoh decoder as well as the directly extracted phrase-tables,
the PhraseTrans SMT systems can fully exploit thepower of the decoder. This led to better performanceeven when the induced phrase-tables were noisy, as
described below.
The relative performance of the PhraseTransSMT systems compared to the Direct SMT systems
was 0.92 to 0.97. These are very promising results. To show how these systems translated thetest sentences, we translated some outputs of the
Spanish-French Direct and PhraseTrans SMT systems into English using the French-English Direct
system. These are shown in Table 3 with the reference English sentences.
The relative performance seems to be related to
the BLEU scores for the Direct SMT systems. It
was relatively high (0.95 to 0.97) for the difficult (interms of BLEU) language pairs but relatively low(0.92) for the easy language pairs; Spanish–Frenchand French–Spanish. There is a lot of room for
improvement for the relatively easy language pairs.
This relationship is stronger than the relationship between the BLEU scores for SE/ET and those for the
PhraseTrans systems, where no clear trend exists.
Table 2 shows the number of phrases stored in thephrase-tables. The Direct SMT systems had 7.3 to
8
A typical reranking approach to SMT (Och et al., 2004)
uses a 1000–best list.

Source–Target
Spanish–French
French–Spanish
German–French
French–German
German–Spanish
Spanish–German

Direct
35.78
34.16
23.37
15.27
22.34
15.50

>
>
>
>
>
>

PhraseTrans32.90 (0.92)
31.49 (0.92)
22.47 (0.96)
14.51 (0.95)
21.76 (0.97)
15.11 (0.97)

>
>
>
>
>
>

SntTrans15
29.49 (0.82)
28.41 (0.83)
22.03 (0.94)
14.03 (0.92)
21.36 (0.96)
14.46 (0.93)

>
>
>
<
>
<

SntTrans1
29.16 (0.81)
27.99 (0.82)
21.64 (0.93)
14.21 (0.93)
20.97 (0.94)
14.61 (0.94)

SE
29.31
27.59
22.40
27.59
22.40
29.31

ET
28.80
29.07
28.80
15.81
29.07
15.81

Table 1: BLEU scores and relative performance

S–F
F–S
G–F
F–G
G–S
S–G

No. of phrases (“M” means 106 )Direct PhraseTrans common
18.2M
190.8M
6.3M
18.2M
186.8M
6.3M
7.3M
174.9M
3.1M
7.3M
168.2M
3.1M
7.5M
179.6M
3.3M
7.6M
176.6M
3.3M

R

P

34.7
34.7
43.2
43.2
44.1
44.1

3.3
3.4
1.8
1.9
1.9
1.9

P=

“S”, “F”, and “G” are the acronyms of Spanish, French, andGerman, respectively. “X–Y” means that “X” is the source language and “Y” is the target language.

Table 2: Statistics for the phrase-tables
18.2 million phrases, and the PhraseTrans systems
had 168.2 to 190.8 million phrases. The numbers ofphrases stored in the PhraseTrans systems were verylarge compared to those of Direct systems.9 However, this does not cause a computational problem in
decoding because those phrases that do not appear insource sentences are filtered so that only the relevantphrases are used during decoding.
The figures in the common column are the numberof phrases common to the Direct and PhraseTranssystems. R (recall) and P (precision) are defined asfollows.
R=

No. of common phrases × 100
No. of phrases in Direct system

9
In Table 2, the PhraseTrans systems have more than 10x
as many phrases as the Direct systems. This can be explained
as follows. Let fi be the fanout of an English phrase i, i.e.,
fi is the number of phrase pairs containing the English phrase
iPin a phrase-table, then the size of the phrase-table is s1 =n
f , where n is the number of distinct English phrases.
i=1 i
When we combine two phrase-tables,
Pn 2the size of the combinedphrase table is roughly s2 =f . Thus, the relative size
i=1 i
2

)
,
of the combined phrase table is roughly r = ss12 = E(f
E(f )
s2
s1
2
where E(f ) = n and E(f ) = n are the averages overfi and fi2 , respectively. As an example, we calculated these
averages for the German-English phrase table. E(f ) was 1.5,E(f 2 ) was 43.7, and r was 28.9. This shows that even if anaverage fanout is small, the size of a combined phrase table canbe very large.

489

No. of common phrases × 100
No. of phrases in PhraseTrans system

Recall was reasonably high. However, the upperbound of recall was 100 percent because we useda multilingual corpus whose sentences were aligned
to each other across all four languages, as describedin Section 4.1. Thus, there is a lot of room for improvement with respect to recall. Precision, on theother hand, was very low. However, translation performance was not significantly affected by this low
precision, as is shown in Table 1. This indicates thatrecall is more important than precision in building
phrase-tables.

5 Related work
Pivot languages have been used in rule-based machine translation systems. Boitet (1988) discusses
the pros and cons of the pivot approaches in multilingual machine translation. Schubert (1988) argues
that a pivot language needs to be a natural language,
due to the inherent lack of expressiveness of artificial languages.
Pivot-based methods have also been used in otherrelated areas, such as translation lexicon induction (Schafer and Yarowsky, 2002), word alignment(Wang et al., 2006), and cross language information
retrieval (Gollins and Sanderson, 2001). The translation disambiguation techniques used in these studies could be used for improving the quality of phrasetranslation tables.
In contrast to these, very little work has been
done on pivot-based methods for SMT. Kauers etal. (2002) used an artificial interlingua for spoken
language translation. Gispert and Mariño (2006)
created an English-Catalan parallel corpus by automatically translating the Spanish part of an EnglishSpanish parallel corpus into Catalan with a SpanishCatalan SMT system. They then directly trained an
SMT system on the English-Catalan corpus. They

showed that this direct training method is superior
to the sentence translation strategy (SntTrans1) intranslating Catalan into English but is inferior to
it in the opposite translation direction (in terms of
the BLEU score). In contrast, we have shown thatthe phrase translation strategy consistently outperformed the sentence translation strategy in the controlled experiments.

6

Conclusion

We have compared two types of pivot strategies,
namely phrase translation and sentence translation.
The phrase translation strategy directly constructs aphrase translation table from a source language andEnglish phrase-table and a target language and English phrase-table. It then uses this phrase table ina phrase-based SMT system. The sentence translation strategy first translates a source language sentence into n English sentences and translates these n
sentences into target language sentences separately.
Then, it selects the highest scoring sentence from thetarget language sentences.
We conducted controlled experiments using theEuroparl corpus to compare the performance of
these two strategies to that of directly trained SMT
systems. The experiments showed that the performance of the phrase translation strategy was statistically significantly better than that of the sentencetranslation strategy and that its relative performancecompared to the directly trained SMT systems was
0.92 to 0.97. These are very promising results.
Although we used the Europarl corpus for controlled experiments, we intend to use the pivot strategies in situations where very limited amount of parallel corpora are available for a source and target language but where relatively large parallel corpora are
available for the source language–English and thetarget language–English. In future work, we willfurther investigate the pivot strategies described inthis paper to confirm that the phrase translation strategy is better than the sentence translation strategy inthe intended situation as well as with the Europarl
corpus.1010

As a first step towards real situations, we conducted additional experiments. We divided the training corpora in Section
4 into two halves. We used the first 292915 sentences to train
source-English SMT systems and the remaining 292915 ones
to train English-target SMT systems. Based on these source-

490

References
Christian Boitet. 1988. Pros and cons of the pivot andtransfer approaches in multilingual machine translation. In Dan Maxwell, Klaus Schubert, and Toon
Witkam, editors, New Directions in Machine Translation. Foris. (appeared in Sergei Nirenburg, Harold
Somers and Yorick Wilks (eds.) Readings in MachineTranslation published by the MIT Press in 2003).
Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della
Pietra, and Robert L. Mercer. 1993. The mathematics
of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2):263–311.
Chris Callison-Burch, Philipp Koehn, and Miles Osborne. 2006a. Improved statistical machine translation using paraphrases. In NAACL.
Chris Callison-Burch, Miles Osborne, and Philipp
Koehn. 2006b. Re-evaluating the role of BLEU inmachine translation research. In EACL.
Adriá de Gispert and José B. Mari no. 2006. CatalanEnglish statistical machine translation without parallel
corpus: Bridging through Spanish. In Proc. of LREC
5th Workshop on Strategies for developing MachineTranslation for Minority Languages.
Tim Gollins and Mark Sanderson. 2001. Improving
cross language information retrieval with triangulatedtranslation. In SIGIR.
Manuel Kauers, Stephan Vogel, Christian Fügen, andAlex Waibel. 2002. Interlingua based statistical machine translation. In ICSLP.
Philipp Koehn and Christof Monz. 2006. Manual and automatic evaluation of machine translation between european languages. In Proceedings on the Workshop on
Statistical Machine Translation, pages 102–121, NewYork City, June. Association for Computational Linguistics.
Philipp Koehn, Franz Josef Och, and Daniel Marcu.
2003. Statistical phrase-based translation. In HLTNAACL.
English and English-target SMT systems, we trained PhraseTrans and SntTrans1 SMT systems. Other experimental conditions were the same as those described in Section 4. The tablebelow shows the BLUE scores of these SMT systems. It indicates that the PhraseTrans systems consistently outperformed
the SntTrans1 systems.
Source-Target
PhraseTrans SntTrans1
Spanish-French
31.57
28.36
French-Spanish
30.18
27.75
German-French
20.48
19.83
French-German
14.38
14.11
German-Spanish
19.58
18.67
Spanish-German
14.80
14.46

Ref
Dir
PT
Ref
Dir
PT
Ref
Dir
PT

i hope with all my heart , and i must say this quite emphatically , that an opportunity will arise when thisdocument can be incorporated into the treaties at some point in the future .
i hope with conviction , and put great emphasis , that again is a serious possibility of including this in the treaties .
i hope with conviction , and i very much , insisted that never be a serious possibility of including this in thetreaties .
should this fail to materialise , we should not be surprised if public opinion proves sceptical about europe , or evenrejects it .
otherwise , we must not be surprised by the scepticism , even the rejection of europe in the public .
otherwise , we must not be surprised by the scepticism , and even the rejection of europe in the public .
the intergovernmental conference - to address a third subject - on the reform of the european institutions is also ofdecisive significance for us in parliament .
the intergovernmental conference - and this i turn to the third issue on the reform of the european institutions is ofenormous importance for the european parliament .
the intergovernmental conference - and this brings me to the third issue - on the reform of the european institutions
has enormous importance for the european parliament .

Table 3: Reference sentences (Ref) and the English translations (by the French-English Direct system) of
the outputs of the Spanish-French Direct and PhraseTrans SMT systems (Dir and PT).
Philipp Koehn, Amittai Axelrod, Alexandra Birch
Mayne, Chris Callison-Burch, Miles Osborne, andDavid Talbot. 2005. Edinburgh system description
for the 2005 IWSLT speech translation evaluation. In
IWSLT.
Philipp Koehn. 2004. Pharaoh: a beam search decoder
for phrase-based statistical machine translation models. In AMTA.

Franz Josef Och, Daniel Gildea, Sanjeev Khudanpur,
Anoop Sarkar, Kenji Yamada, Alex Fraser, Shankar
Kumar, Libin Shen, David Smith, Katherine Eng,
Viren Jain, Zhen Jin, and Dragomir Radev. 2004. A
smorgasbord of features for statistical machine translation. In HLT-NAACL.
Franz Josef Och. 1999. An efficient method for determining bilingual word classes. In EACL.

Philipp Koehn. 2005. Europarl: A parallel corpus forstatistical machine translation. In MT Summit.

Franz Josef Och. 2003. Minimum error rate training instatistical machine translation. In ACL.

Dragos Stefan Munteanu and Daniel Marcu. 2005. Improving machine translation performance by exploiting non-parallel corpora. Computational Linguistics,
31(4):477–504.

Kishore Papineni, Salim Roukos, Todd Ward, and WeiJing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In ACL.

Makoto Nagao. 1981. A framework of a mechanical translation between Japanese and English by analogy principle. In the International NATO Symposium
on Artificial and Human Intelligence. (appeared inSergei Nirenburg, Harold Somers and Yorick Wilks(eds.) Readings in Machine Translation published by
the MIT Press in 2003).

Philip Resnik and Noah A. Smith. 2003. The web
as a parallel corpus. Computational Linguistics,
29(3):349–380.
Charles Schafer and David Yarowsky. 2002. Inducing translation lexicons via diverse similarity measuresand bridge languages. In CoNLL.
Klaus Schubert. 1988. Implicitness as a guiding principle in machine translation. In COLING.

Sonja Nießen and Hermann Ney. 2004. Statistical machine translation with scarce resources using morphosyntactic information. Computational Linguistics,
30(2):181–204.

Andreas Stolcke. 2002. SRILM - an extensible language
modeling toolkit. In ICSLP.

Franz Josef Och and Hermann Ney. 2002. Discriminative training and maximum entropy models for statistical machine translation. In ACL.

Masao Utiyama and Hitoshi Isahara. 2003. Reliablemeasures for aligning Japanese-English news articles
and sentences. In ACL, pages 72–79.

Franz Josef Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19–51.

Haifeng Wang, Hua Wu, and Zhanyi Liu. 2006. Wordalignment for languages with scarce resources usingbilingual corpora of other language pairs. In COLING/ACL 2006 Main Conference Poster Sessions.

Franz Josef Och and Hermann Ney. 2004. The alignment template approach to statistical machine translation. Computational Linguistics, 30(4):417–449.

491