On the Importance of Pivot Language Selection
for Asian Language Translation
Michael Paul∗† , Hirofumi Yamamoto†‡ , Eiichiro Sumita† and Satoshi Nakamura†† NICT/ATR, Hikaridai 2-2-2, Keihanna Science City, 619-0288 Kyoto, Japan‡ Kinki University School of Science and Engineering, Higashi-Osaka City, 577-8502, Japan
Email: Michael.Paul@nict.go.jp

Abstract—Recent research on multilingual statistical machinetranslation focuses on the usage of pivot languages in order toovercome resource limitations for certain language pairs. Dueto the richness of available language resources, English is in
general the pivot language of choice. In this paper, we investigate
the appropriateness of Asian languages as pivot languages.
Experimental results using state-of-the-art statistical machinetranslation techniques to translate between eight Asian languages
revealed that the translation quality of 73% of the language pairs
improved when a non-English pivot language was chosen.

I. I NTRODUCTION

closeness between specific language pairs to achieve highquality translation hypotheses in the first step to minimize
the detoriation effects of the pivot approach. For example, [2]proposes a method to translate Catalan-to-English via Spanish.
However, both of the above criteria might not be sufficient tochoose the best pivot language, especially for Asian languages
where, besides for Chinese, only few parallel text corpora for
Asian languages and English are publicly available. Moreover,language families in Asia are quite diverse.
This paper investigates the appropriateness of Asian languages as pivot languages to support future research onmachine translation between Asian languages. Pivot translation
experiments using state-of-the-art statistical machine translation techniques to translate between eight Asian languages(Chinese, Hindi, Indonesian, Japanese, Korean, Malay, Thai,
Vietnamese) are carried out and the effects of selecting a nonEnglish language as the pivot language are compared towards
the English pivot approach.
Section II gives an overview on recent machine translation
research efforts in Asia and summarizes the availability of
Asian language resources. Section III introduces the pivot
translation approach within the framework of statistical machine translation. Section IV outlines the pivot translation
experiments, discusses the effects of using non-English pivot
languages and identifies the best-suited pivot language fortranslations between Asian languages.

The translation quality of state-of-the-art phrased-based statistical machine translation (SMT) approaches heavily depends
on the amount of bilingual language resources available totrain the statistical models. For frequently used language
pairs like French-English or Chinese-English, large-sized text
data sets are readily available. There exist several international data collection initiatives like the Linguistic Data Consortium (LDC, http://www.ldc.upenn.edu), the European Language Resource Association (ELRA, http://www.elra.info), or
GSK (http://www.gsk.or.jp/catalog.html) amassing and distributing large amounts of textual data. However, for less frequently
used language pairs, e.g., most of the Asian languages, only
a limited amount of bilingual resources are available, if at all.
In order to overcome such language resource limitations,
recent research on multilingual statistical machine translation
focuses on the usage of pivot languages [1]. Instead of aII. M ACHINE T RANSLATION FOR A SIAN L ANGUAGES
direct translation between two languages where only a limited
amount of bilingual resources is available, the pivot transThe Asia-Pacific Association for Machine Translation
lation approach makes use of a third language that is more (AAMT, http://www.aamt.info) lists a variety of machine transappropriate due to the availability of more bilingual corpora lation products and machine translation services translating
and/or its relatedness towards either the source or the target from/to Asian languages including Hindi (hi), Indonesianlanguage. In a first step, the source language input is translated (id), Japanese (ja), Korean (ko), Malay (ms) Thai (th),into the pivot language using statistical translation models Vietnamese (vi), and Chinese (zh). Table I summarizes thetrained on the source-pivot language resources. In the second amount of publicly available translation systems/services forstep, the obtained pivot language result is translated into the the respective language pairs. Most of the MT products andtarget language using a second translation engine trained on services focus on Japanese ⇔ English translations, followed
the pivot-target language resources.
by Japanese ⇔ Korean and Chinese ⇔ English. However, only
In previous research on pivot translation approaches, the a few MT tools and MT research prototypes are available
pivot language was mainly selected based on (a) the avail- for language translations involving Hindi, Indonesian, Malay,ability of bilingual language resources or (b) language re- Thai, or Vietnamese [3].
latedness between source/pivot languages. For most recent
Similarly, the amount of publicly available bilingual text
research efforts, English is the pivot language of choice due corpora is quite limited. Table II summarizes the amount ofto the richness of available language resources. For exam- bilingual text corpora or dictionaries available from LDC,
ple, [1] exploits the Europarl corpus for comparing pivot ELRA, GSK and other publicly available multilingual resources
translation approaches between French/German/Spanish via used in recent MT evaluation campaigns like NIST or IWSLT.
English. Moreover, several research efforts tried to exploit
Most language resources are available for Chinese ⇔ English
－ the
646 －

TABLE I
A SIAN L ANGUAGE MT P RODUCTS /S ERVICES
en
hi
id
ja
ko
ms
th
vi
zh

en
–
4
5
89
20
2
4
3
34

hi
4
–
3
3
3
1
1
3
3

id ja ko
5 92 243 3 3– 3 3
3 – 393 36 –
1 1 1
1 1 1
3 3 3
3 30 9

ms
2
1
1
1
1
–
1
1
1

th
3
1
1
4
1
1
–
1
1

vi
3
3
3
3
3
1
1
–
3

zh
34
3
3
32
9
1
1
3
–

TABLE IIA SIAN L ANGUAGE R ESOURCES
en
en 132
hi

hi
2
3
id

id ja ko
1 11 6
ja 13 ko 14
ms

ms
3
1
2
th

th
1
1
1
1
vi

vi zh
- 72
1 1
1
1
zh 64

are combined to a new SRC-TRG phrase-table bymerging SRC-PVT and PVT-TRG phrase-table entries
with identical pivot language phrases and multiplying
posterior probabilities [1], [6].
4) bridging at translation time where the coupling is integrated into the SMT decoding process by modeling thepivot text as a hidden variable and assuming independence between source and target sentences [7].
In order to investigate the effects of the pivot language
selection for statistical machine translation involving Asian
languages, the simplest method of cascading two SMT systems
is exploited in the pivot translation experiments reported in
Section IV.
IV. E XPERIMENTS

The effects of pivot language selection are investigatedusing the Basic Travel Expressions Corpus (BTEC), which isa collection of sentences that bilingual travel experts consider
useful for people going to or coming from another country.
The corpus currently covers 18 of the major world languages[5]. Besides English, we selected the eight Asian languages
listed in Section II for the pivot translation experiments.
These languages differ largely in word order (SVO, SOV),
and Japanese ⇔ English. However, almost no bilingual data is segmentation unit (phrase, word, none), and morphology (poor,
medium, rich). Concerning word segmentation, the corpora
available for the other language pairs.
In order to fill the gap, recent research activities on spoken were preprocessed using word-segmentation tools for lanlanguage translation conducted by the Asian Speech Trans- guages that do not use white-space to separate word tokens
lation Consortium (ASTAR, http://www.slc.atr.jp/AStar), which like Chinese, Japanese, Korean, and Thai. All data sets wereis an international partnership of research laboratories that case-sensitive with punctuation marks preserved.
However, in a real-world application, identical language
focuses on the development of large-scaled spoken language
corpora in Asia. As a first result of these activities, a Japanese- resources covering three or more languages are not necessarily
English speech corpus [4] comprising tourism-related sen- to be expected. In order to avoid a trilingual scenario fortences has been translated into the native languages of the the pivot translation experiments described in this paper, the
ASTAR members resulting in a multilingual sentence-aligned full BTEC corpus consisting of 160k sentence-aligned data
corpus [5]. This corpus is introduced in detail in Section IV sets was randomly split into two subsets of 80k sentences
and is exploited to investigate the effects of pivot language each, whereby the first set of sentence pairs was used to train
the source-to-pivot translation models (80k sp ) and the second
selection for Asian languages.
subset of sentence pairs was used to train the pivot-to-target
III. P IVOT T RANSLATION
translation models (80k pt ).
Pivot translation is a translation from a source language
The characteristics of the utilized BTEC corpus data sets are(SRC) to a target language (TRG) through an intermediate summarized in Table III. The sentence length is given as thepivot (or bridging) language (PVT). Within the SMT frame- average number of words per sentence. In order to get an idea
work, the following coupling strategies have already been of how difficult the translation task for the different languages
investigated:
is supposed to be, we calculated the language perplexity of the1) cascading of two translation systems where the first MT target language evaluation data sets according to a standard 5engine translates the source language input into the pivot gram language model trained on the respective training data
language and the second MT engine takes the obtained sets. For each language, the language perplexity and the total
pivot language output as its input and translates it into entropy, i.e., the entropy multiplied by the number of words ofthe target language.
the evaluation data set, are listed. The higher the total entropy,2) pseudo corpus approach that (a) creates a “noisy” SRC- the more difficult the translation task is supposed to be.
TRG parallel corpus by translating the pivot language
For the training of the statistical models, standard wordparts of the SRC-PVT training resources into the target alignment (GIZA++ [8]) and language modeling (SRILM [9])language using an SMT engine trained on the PVT-TRG tools were used. For translation, the Cleopatra decoder, an
language resources and (b) directly translates the source in-house phrase-based SMT decoder [10], was used.
language input into the target language using a single
For the evaluation of translation quality, a test data set
SMT engine that is trained on the obtained SRC-TRG consisting of 510 sentences of the BTEC corpus reserved for
language resources [2].
evaluation purposes was translated by the respective translation
3) phrase-table composition in which the translation mod- engines and evaluated using the standard automatic evaluationels of the SRC-PVT and PVT-TRG translation engines
metrics BLEU [11] and METEOR [12]. For the experimental－ 647 －TABLE III
P IVOT T RANSLATION L ANGUAGE R ESOURCES

BTEC
Corpus# of sentences
en vocabulary
avg. length
hi vocabulary
avg. length
id vocabulary
avg. length
ja vocabulary
avg. length
ko vocabulary
avg. length
ms vocabulary
avg. length
th vocabulary
avg. lengthvi vocabulary
avg. lengthzh vocabulary
avg. length

training
80ksp 80kpt
80,000 80,000
12,264 11,129
7.8
7.1
20,725 12,320
8.4
7.5
14,585 13,343
7.0
6.5
13,868 12,621
8.8
8.2
13,546 12,381
8.3
7.8
15,113 13,752
7.1
6.6
6,103 5,637
8.1
7.5
7,980 7,388
9.4
8.7
11,084 10,220
6.6
6.6

eval
perplexity
set(total entropy)
510
–
896
25.6
7.5
(17899.0)
1162
104.9
8.1
(27644.8)
1,000
50.2
7.1
(20467.0)
959
18.1
8.5
(18136.4)
946
17.8
8.1
(17135.1)
995
50.9
7.1
(20402.1)
738
36.5
7.7
(20516.8)
884
26.1
9.2
(22043.2)
908
28.3
7.0
(17136.2)

TABLE IV
AUTOMATIC E VALUATION M ETRICS

BLEU:

the geometric mean of n-gram precision by the system output with respect to reference translations.
Scores range between 0 (worst) and 1 (best) [11]METEOR: a metric that calculates unigram overlaps between
translation and reference texts taking into accountvarious levels of matches (exact, stem, synonym).
Scores range between 0 (worst) and 1 (best) [12]results in this paper, the given scores are calculated as theaverage of the respective BLEU and METEOR scores obtainedfor each system output.
A. Direct Translation

B. Pivot Translation
The automatic evaluation scores for all pivot translation
language-pair combinations (SRC-PVT-TRG) are summarizedin Table VI whereby for each source-target language pair,
the results of the pivot translation experiments using (a)English(en) and (b) the best performing language (best) as
the pivot language are listed. The experimental results showthat English is indeed the best pivot language when translatingfrom Hindi or Malay into most of the other target languages.
However, for Korean and Chinese as the source language,
significantly lower evaluation scores were obtained for English compared to several other Asian pivot languages for alltranslation directions. In the case of Korean, Japanese is thepivot language of choice when translating into other Asian
languages, and vice versa. For Chinese, Indonesian, Thai, andVietnamese, the optimal pivot language depends largely on therespective target language.
However, the selection of the optimal pivot language is not
symmetric for most of the language pairs. Quite a different
picture is obtained when analyzing the results according to
the respective target language. For translations into Hindi,
either Japanese or Korean should be preferred towards English
as the pivot language. In the case of Malay as the target
language, Indonesian outperforms all other pivot languages.
Nevertheless, the English pivot approach seems to be effective
for most translations into Korean and Japanese.
Comparing the results of the pivot translation experiments
towards the direct translation results, we can see that for
43% of the language pairs, the pivot approach outperforms
the direct translation approach significantly when the optimal
pivot language is selected. This phenomena is caused mainlyby (a) the unrelatedness between SRC and TRG (18×), (b) the
closeness between PVT and TRG (3×) and (c) the closenessbetween SRC and PVT (1×).

The automatic evaluation scores for all source and target C. Pivot Language Selection
Besides the automatic evaluation scores, Table VI lists also
language pair combinations of the direct translation approachtheoptimal pivot language for each source-target language
are summarized in Table V. For each target language, thepair
in boldface. Moreover, for each language, the amounthighest evaluation scores are marked in boldface.
oflanguagepairs that achieved the highest scores using thisDespite slight score differences, quite similar levels of
languageasthepivot is given in Table VII. The experimental
translation quality were obtained by the SRC-PVT and theresults
show
that
the English pivot approach still achieves the
PVT-TRG models for most of the language pairs. The highest
highestscoresforthe majority of the examined language pairs.
evaluation scores were achieved for closely related language
However,in73%
ofthe cases, an Asian pivot language, mainly
pairs like Japanese ⇔ Korean and Indonesian ⇔ Malay. In
Japanese,
Malay,
Indonesian,and Korean is to be preferred.
addition, relatively high translation quality was achieved forJapanese ⇔ Chinese. As indicated by the total entropy figures
V. C ONCLUSION
in Table III, the hardest translation tasks were those translating
In this paper, the effects of using Asian pivot languages
to Hindi followed by Malay, Thai, and Indonesian.
Interestingly, all language pairs having English as the source for translations between eight Asian languages were comlanguage achieved not always the highest scores, but per- pared to the standard English pivot translation approach. The
formed better than most of the remaining languages which experimental results revealed that English was indeed moreis in contrast to the language pairs translating into English frequently (26.8% out of 56 language pairs) selected as thewhere only mid-level scores were achieved. This indicates, best pivot language over any other Asian language. However,that a relatively larger detoriation in translation quality is to its usage is mainly limited to translations from Hindi andbe expected for the SRC-PVT translation step when English is Malay or translations into Korean. Otherwise, the English
used as the pivot language compared to other pivot languages pivot approach is significantly outperformed by using Asian
where higher evaluation scores for translations into the pivot languages as the pivot languages, especially Japanese, Malay,
Indonesian, or Korean. In contrast to previous research on－ 648 －
language were obtained.

TABLE V
T RANSLATION Q UALITY OF D IRECT T RANSLATION A PPROACHESSRC\TRG

en
hi
id
ja
ko
ms
th
vi
zh

en
–
0.49060.6526
0.67360.65380.64050.6021
0.63200.6513hi
0.4370–0.41350.42240.42770.39000.3946
0.40600.4029id
0.64710.4695–0.63920.62080.79070.6024
0.59340.6238ja
0.81570.55420.7233–0.87630.71010.69690.7026
0.7891ko
0.72140.42100.62180.8460–0.61510.62050.61710.7371ms
0.62150.47190.78980.59800.5921–0.56470.56950.5797th
0.64230.41750.5956
0.61470.61070.5940–0.58970.6047vi
0.70450.51740.6556
0.68880.67080.6386
0.6768–0.6806zh
0.71010.47140.62030.78170.76750.61100.65570.6241–TABLE VIT RANSLATION Q UALITY OF P IVOT T RANSLATION A PPROACHESSRC PVT

hi
id
ja
ko
ms
th
vi
zh

en
best
en
best
en
best
en
best
en
best
en
best
en
best
en
best

hi
id
ja
ko
ms
th
vi
zh
–
0.51050.6261
0.52480.49730.43010.5352
0.5240– (en) 0.5105 (ms) 0.6287 (en) 0.5248 (en) 0.4973 (en) 0.4301 (en) 0.5352 (en) 0.5240
0.4102–0.72510.64510.61350.56920.62560.6223(ja) 0.4221– (en) 0.7251 (en) 0.6451 (vi) 0.6227 (ms) 0.5858 (ms) 0.6563 (ja) 0.6447
0.42170.6043–0.71890.5952
0.5864
0.66850.6964(ko) 0.4250 (ms) 0.6160– (zh) 0.7286 (id) 0.6119 (ko) 0.6079 (en) 0.6685 (ko) 0.7729
0.41450.5871
0.7612–0.57180.57270.65360.6948(ja) 0.4266 (ja) 0.6207 (zh) 0.7739– (id) 0.5994 (ja) 0.6123 (ja) 0.6802 (ja) 0.7499
0.4234
0.63250.73130.6301–
0.56870.61880.6182(en) 0.4234 (en) 0.6325 (en) 0.7313 (en) 0.6301– (id) 0.5789 (id) 0.6372 (ko) 0.6254
0.40310.59030.71030.61410.5738–0.63190.6234(vi) 0.4062 (en) 0.5903 (ms) 0.7236 (zh) 0.6242 (id) 0.5803– (zh) 0.6446 (vi) 0.6401
0.39070.57070.72620.64770.55080.5504–0.6266(ko) 0.4014 (ms) 0.5956 (th) 0.7295 (en) 0.6477 (id) 0.5614 (ms) 0.5588– (th) 0.6454
0.40390.59350.75650.67940.56180.57050.6473–(ko) 0.4191 (ms) 0.6062 (ko) 0.7841 (ja) 0.7140 (id) 0.5866 (ja) 0.6032 (th) 0.6568–
TABLE VIIP IVOT L ANGUAGE S ELECTION

PVT

en
ja
ms
id

usage (%)
15 (26.8)
9(16.1)
8(14.3)
7(12.5)

PVT

ko
zh
th
vi

usage (%)
7 (12.5)
4
(7.2)
3
(5.3)
3
(5.3)

between Asian languages”, the Special Coordination Funds
for Promoting Science and Technology of the Ministry of
Education, Culture, Sports, Science and Technology, Japan.
R EFERENCES[1] M. Utiyama and H. Isahara, “A comparison of pivot methods for phrasebased statistical machine translation,” in Proc. of HLT, New York, USA,2007, pp. 484–491.
pivot translation approaches, the most prominent criteria on[2] A. Gispert and J. Marino, “Catalan-english statistical machine translation
without parallel corpus: bridging through spanish,” in Proc. of 5th LREC,
how to select the best pivot language was not the language
Genoa, Italy, 2006, pp. 65–68.
relatedness between source and pivot languages, but the[3] H. Isahara et al (Ed.), “Special Issue on Machine Translation Activities
unrelatedness between source and target languages. In such ain Asia,” AAMT Journal, pp. 1–40, 2005.
[4] G. Kikui, S. Yamamoto, T. Takezawa, and E¿ Sumita, “Comparative
case, the pivot language approach frequently achieved a better
study on corpora for speech translation,” IEEE Transactions on Audio,translation quality than the one of the direct translation of the
Speech and Language, vol. 14(5), pp. 1674–1682, 2006.
source language input into the target language, because the[5] M. Paul et al, “Multilingual Mobile-Phone Translation Services,” inProc. of 22nd COLING, Manchester, UK, 2006, pp. 165–168.
translation quality of the source-pivot and pivot-target engines[6] H. Wu and H. Wang, “Pivot Language Approach for Phrase-Basedwere significantly higher than the direct translation engines.
Statistical Machine Translation,” in Proc. of ACL, Prague, Czech
Future research will have to investigate in detail, what kind
Republic, 2007, pp. 856–863.
[7] N. Bertoldi, M. Barbaiani, M. Federico, and R¿ Cattoni, “Phrase-Basedof features are important to select a pivot language for new
Statistical Machine Translation with Pivot Languages,” in Proc. of the
Asian source and target language pairs. Besides the translation
IWSLT, Hawaii, USA, 2008, pp. 143–149.
quality of SMT engines, automatic metrics to measure the[8] F. Och and H. Ney, “A Systematic Comparison of Statistical Alignment
Models,” Computational Linguistics, vol. 29(1), pp. 19–51, 2003.
closeness of a language pair should also be taken into account[9] A. Stolcke, “SRILM - an extensible language modeling toolkit,” into find optimal pivot languages and improve the usability ofProc. of ICSLP, Denver, 2002, pp. 901–904.
machine translation between Asian languages further.
[10] A. Finch et al, “The NICT/ATR Speech Translation System for IWSLT
2007,” in Proc. of the IWSLT, Trento, Italy, 2007, pp. 103–110.
ACKNOWLEDGMENT[11] K. Papineni, S. Roukos, T. Ward, and W. Zhu, “BLEU: a Method forAutomatic Evaluation of Machine Translation,” in Proc. of the 40th
This work is partly supported by the Grant-in-Aid for SciACL, Philadelphia, USA, 2002, pp. 311–318.
entific Research (C) Number 19500137 and ”Construction of [12] S. Banerjee and A. Lavie, “METEOR: An Automatic Metric for MT
speech translation foundation aiming to overcome the barrier－ 649 － Evaluation,” in Proc. of the ACL, Ann Arbor, US, 2005, pp. 65–72.