言語処理学会 第21回年次大会 発表論文集 (2015年3月)

Large-scale Japanese-Chinese Scientific Dictionary Construction via
Pivot-based Statistical Machine Translation
Chenhui Chu1,3 , Raj Dabre1 , Toshiaki Nakazawa1,2 and Sadao Kurohashi1
1Graduate School of Informatics, Kyoto University
2
Japan Science and Technology Agency
3
Japan Society for the Promotion of Science Research Fellow{chu,raj,kuro}@nlp.ist.i.kyoto-u.ac.jp nakazawa@pa.jst.jp

1 Introduction
Pivot-based statistical machine translation (SMT)(Wu and Wang, 2007) has been shown a possible
way of constructing a dictionary for the language
pairs that have scarce parallel data (Tsunakawa et al.,2009). The assumption of this method is that there isa pair of large-scale parallel data: one between thesource language and an intermediate resource richlanguage (henceforth called pivot), and one between
that pivot and the target language. Once this assumption suffices, we can use the source-pivot andpivot-target parallel data to develop a source-target
term1 translation model for dictionary construction.
There are two main advantages of pivot-based
SMT for dictionary construction. One is that because pivot-based SMT uses the log linear model
as conventional phrase-based SMT (Koehn et al.,2007) does, various features can be integrated to improve the accuracy. The other is that this method
can address the data sparseness problem of directly
merging the source-pivot and pivot-target terms, because it can use the portion of terms to generate newterms. However, the potential of this method has
not been fully explored. Small-scale experiments in(Tsunakawa et al., 2009) showed very low accuracyof pivot-based SMT for dictionary construction.2This paper presents our study to construct a largescale Japanese-Chinese (Ja-Zh) scientific dictionary,using large-scale Japanese-English (Ja-En) (49.1Msentences and 1.4M terms) and English-Chinese
1

In this paper, we call the entries in the dictionary terms. Aterm consists of one or multiple tokens.
2
The highest accuracy evaluated based on the 1 best translation is 0.217 in (Tsunakawa et al., 2009).

(En-Zh) (8.7M sentences and 4.5M terms) parallel
data via pivot-based SMT. We generate a large pivot
translation model using the Ja-En and En-Zh parallel data. Moreover, a small direct Ja-Zh translation
model is generated using small-scale Ja-Zh parallel
data (680k sentences and 561k terms). Both the direct and pivot translation models are used to translate the Ja terms in the Ja-En model to Zh, to construct a large-scale Ja-Zh dictionary (about 58.5Mterms). In addition, we exploit linguistic knowledge of common Character characters (Chu et al.,2013) shared in Ja-Zh to further improve the translation model. Large-scale experiments on scientificdomain data indicate that dictionary construction via
pivot-based SMT can achieve high enough accuracy
for practical use.

2 Phrase-based SMT
This section gives a brief overview of phrase-based
SMT (Koehn et al., 2007), which is the foundation of pivot-based SMT. In phrase-based SMT, thetranslation model is represented as a phrase table,
containing phrase pairs together with their featurescores. The phrase pairs are extracted from a parallel corpus based on unsupervised word alignments.
Inverse and direct phrase translation probabilities
ϕ(f |e) and ϕ(e|f ), inverse and direct lexical weighting lex(f |e) and lex(e|f ) are used as features forthe phrase table. Phrase translation probabilities
are calculated via maximum likelihood estimation,
which counts how often a source phrase f is aligned
to target phrase e in the parallel corpus, and viseversa. Lexical weighting is the average word translation probability calculated using internal word align-― 99 ―Copyright(C) 2015 The Association for Natural Language Processing.
All Rights Reserved.

!"#$%&=(,.:+&'%,"-.&+"/0.&!"#$%&:*,';-&'(*)&111&#$%111&2&&!,-.*/&111&(&)%111&2&&&&&&

A*BB*9&A%(9.-.&:%",":+.,-0!"#$%&>(:?*9",<&!"#+*#$%&
34567&!"#$%&111&!"&!"#$%'()*+&
111&!"#$%&&&&!"#89&'%,"-.&+"/0.&!"#$%&111&"="'+.,&111&2&&!"#$%*'(*)&111&+&&"="'+.,&',*+.(9&111&2&&&&&&!"#89&:*,';-&":<0&111&&'&111&2&&"="'+.,&111&!"&111&2&&&&&&

89#$%&:*,';-&

3()*?9@&89#$%&'%,"-.&+"/0.&!"#$%&111&!"&111&2&&&&&&!"#$%&'()*+&'%,"-.&+"/0.&

Figure 1: Overview of our dictionary construction method.

ments of a phrase pair, which is used to smooththe overestimation of the phrase translation probabilities. Other typical features such as the reordering model features and the n-gram language modelfeatures are also used in phrase-based SMT. Thesefeatures are combined in a log linear model, andtheir weights are tuned using a small size of parallel
sentences. During decoding, these features together
with their tuned weights are used to produce newtranslations.

generating the inverse and direct phrase translation
probabilities ϕ(f |e) and ϕ(e|f ), inverse and direct
lexical weighting lex(f |e) and lex(e|f ) for the generated source-target phrase pairs using the pivotsare:∑ϕ(f |e) =ϕ(f |pi ) ∗ ϕ(pi |e)(1)

ϕ(e|pi ) ∗ ϕ(pi |f )(2)lex(f |pi , a1 ) ∗ lex(pi |e, a2 )(3)lex(e|pi , a2 ) ∗ lex(pi |f, a1 )

(4)

pi

ϕ(e|f ) =lex(f |e, a) =∑∑pipi3Dictionary Construction via Pivot-based
SMT

Figure 1 illustrates an overview of our construction
method. We first generate Ja-Zh (source-target), JaEn (source-pivot) and En-Zh (pivot-target) phrase
tables from parallel data respectively. The generated
Ja-Zh phrase table is used as the direct table. Using
the Ja-En and En-Zh phrase tables, we construct aJa-Zh pivot phrase table via En. The direct and pivot
tables are then combined and used for phrase-based
SMT to translate the Ja phrases in the Ja-En phrase
table to Zh, to construct a large-scale Ja-Zh dictionary. In addition, we use common Chinese characters to generate Chinese character features for thephrase tables to improve the SMT performance.

lex(e|f, a) =∑pi

where a1 is the alignment between phrases f(source) and pi (pivot), a2 is the alignment between
pi and e (target) and a is the alignment between eand f .
3.2

Combination of the Direct and Pivot
Phrase Tables

To combine the direct and pivot phrase tables, wemake use of the multiple decoding paths (MDP)
method of the phrase-based SMT toolkit Moses(Koehn et al., 2007). MDP uses both the tables simultaneously while decoding. Translation options
are collected from both the tables, which allows formore translation options.

3.1 Pivot Phrase Table Generation
We follow the phrase table triangulation method(Wu and Wang, 2007) to generate the pivot phrase
table. This method generates a source-target phrase
table via all their shared pivot phrases in the sourcepivot and pivot-target tables. The formulae for

3.3 Chinese Character Features
Ja-Zh shares Chinese characters. Because many
common Chinese characters exist in Ja-Zh, they
have been shown to be very effective in many JaZh natural language processing (NLP) tasks (Chu et― 100 ―Copyright(C) 2015 The Association for Natural Language Processing.
All Rights Reserved.

al., 2013). In this paper, we compute Chinese character features for the phrase pairs in the translation
models, and integrate these features in the log-linear
model for decoding. In detail, we compute following two features for each phrase pair:Ja CC num + Zh CC num(5)
Ja char num + Zh char num
Ja CCC num + Zh CCC num
CCC ratio =(6)Ja CC num + Zh CC num

4.2 Settings
We compared following two training data settings:• Dic: Only use the dictionaries for training.
• Corpus+Dic: Use both the dictionaries and corpora for training.

CC ratio =where char num, CC num and CCC num denote the number of characters, Chinese charactersand common Chinese characters in a phrase respectively. The common Chinese character ratio is calculated based on the Chinese character mapping tablein (Chu et al., 2013).

4In addition, we compared following three methods
for training the translation model:• Direct: Only use the Ja-Zh data to train a direct
Ja-Zh model.
• Pivot: Use the Ja-En and En-Zh data for training Ja-En and En-Zh models, and construct apivot Ja-Zh model using the phrase table triangulation method.

Experiments

We conducted experiments to show the effectivenessof our dictionary construction method. The accuracyof our method was evaluated using manually createddictionaries.
4.1 Data
4.1.1 Training data
We used following two types of training data:• Bilingual dictionaries: we used the scientific
Ja-En, En-Zh and Ja-Zh dictionaries provided
by the Japan Science and Technology Agency(JST)3 and the Institute of Science and Technology information of China (ISTIC),4 containing 1.4M , 4.5M and 561k term pairs respectively.
• Parallel corpora: the scientific Ja-En, En-Zh
and Ja-Zh corpora we used were also provided
by JST and ISTIC, containing 49.1M , 8.7M
and 680k sentence pairs respectively.

• Direct+Pivot: Combine the direct and pivot JaZh models using MDP.
For the models trained on Corpus+Dic, we also conducted experiments additionally using the Chinesecharacter features (labeled +CC). For decoding, weused Moses (Koehn et al., 2007) with the default options. We trained a word 5-gram language modelon the Zh side of all the En-Zh and Ja-Zh training
data (14.4M sentences) using the SRILM toolkit6
with interpolated Kneser-Ney discounting. Tuning
was performed by minimum error rate training, andit was re-run for every experiment.
4.3 Evaluation Criteria
Following (Tsunakawa et al., 2009), we evaluatedthe accuracy on the test set using the following threemetrics:

4.1.2 Tuning and Testing dataWe used the terms with two reference translations5 in the Ja-Zh Iwanami biology dictionary(5,890 pairs) and the Ja-Zh life science dictionary(4,075 pairs) provided by JST. Half of the data ineach dictionary was used for tuning (4,983 pairs),
and the other half for testing (4,982 pairs).
3

http://www.jst.go.jp
http://www.istic.ac.cn
5Different terms are annotated with different number of reference translations in these two dictionaries.
4― 101 ―• 1 best: Percentage of terms where the top 1
translation given by the MT system is the correct one.
• 20 best: Percentage of terms where the correcttranslation is included in the top 20 translations
given by the MT system.
• Mean Reciprocal Rank (MRR): Let w be asource term, rankw denotes the rank of its correct translation within the list of translations
given by the MT system, V denotes the set of6

http://www.speech.sri.com/projects/srilm

Copyright(C) 2015 The Association for Natural Language Processing.
All Rights Reserved.

Training data
Dic

Corpus+Dic

MethodDirect
Pivot
Direct+Pivot
DirectDirect+CC
Pivot
Pivot+CC
Direct+Pivot
Direct+Pivot+CCBLEU-4
31.22
45.32
46.24
40.64
40.84
52.11
53.24
53.54
54.17

Accuracy w/ OOV
1 best 20 best MRR
0.2912 0.3792 0.3239
0.4123 0.6202 0.4836
0.4213 0.6204 0.4925
0.3697 0.5255 0.4258
0.3721 0.5255 0.4271
0.4914 0.7252 0.5712
0.4984 0.7258 0.5766
0.5024 0.7377 0.5875
0.5157 0.7356 0.5950

OOV term
37%
18%
18%
26%
26%
9%
9%
8%
8%

Accuracy w/o OOV
1 best 20 best MRR
0.4476 0.5833 0.4978
0.5022 0.7559 0.58930.5115 0.7542 0.5983
0.4978 0.7082 0.5736
0.5011 0.7082 0.5754
0.5371 0.7927 0.6243
0.5448 0.7933 0.6302
0.5485 0.8054 0.6414
0.5630 0.8032 0.6496

Table 1: Evaluation results.

terms used for evaluation. Then MRR is defined as:
M RR =1 ∑1
|V |
rankw

(7)

As there are 636M unique Ja phrases in the Ja-En
Corpus+Dic phrase table and we suppose that 10%of them are terms, we are able to construct a largescale Ja-Zh dictionary containing 58.5M 8 terms.

w∈V

We used the top 20 translations given by theMT system for calculating MRR.
In addition, we report the BLEU-4 scores that werecomputed on the word level.
4.4 Results
Table 1 shows the evaluation results. We also show
the percentage of out-of-vocabulary (OOV) terms,7and the accuracy with and without OOV terms respectively. In general, we can see that Pivot performs better than Direct, because the data of Ja-En
and En-Zh is larger than that of Ja-Zh. Direct+Pivot
shows better performance than either method. Training on more data of Corpus+Dic is better than Dic
only. Chinese character features can further improve
the accuracy.
However, the accuracy of our best performing
method Direct+Pivot+CC (Corpus+Dic) is still nothigh enough according to our evaluation method.
We manually investigated the terms, whose top 1
translation was evaluated as incorrect according toour evaluation method. Based on our investigation,nearly 75% of them were actually correct translations. They were undervalued because they are notcovered by the reference translations in our test set.
Taking this observation into consideration, the actual 1 best accuracy is about 0.9. Automatic evaluation tends to greatly underestimate the results because of the incompleteness of the test set.
75 Conclusion and Future WorkIn this paper, we presented a dictionary construction method via pivot-based SMT. Large-scale JaZh experiments showed the effectiveness of ourmethod. In the future, we plan to further improve
the accuracy of our method by significance testing
based pruning and integrating contextual features estimated on the source and target corpora.

References
Chenhui Chu, Toshiaki Nakazawa, Daisuke Kawahara,and Sadao Kurohashi. 2013. Chinese-japanese machine translation exploiting chinese characters. ACMTransactions on Asian Language Information Processing (TALIP), 12(4):16:1–16:25.
Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris
Callison-Burch, Marcello Federico, Nicola Bertoldi,
Brooke Cowan, Wade Shen, Christine Moran, RichardZens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source
toolkit for statistical machine translation. In Proceedings of ACL, pages 177–180.
Takashi Tsunakawa, Naoaki Okazaki, Xiao Liu, andJun’ichi Tsujii. 2009. A chinese-japanese lexicalmachine translation through a pivot language. ACMTransactions on Asian Language Information Processing (TALIP), 8(2):9:1–9:21, May.
Hua Wu and Haifeng Wang. 2007. Pivot language
approach for phrase-based statistical machine translation. Machine Translation, 21(3):165–181, September.
8An OOV term contains at least one OOV word.

― 102 ―636M ∗ 10% ∗ (1 − 8%) = 58.5M

Copyright(C) 2015 The Association for Natural Language Processing.
All Rights Reserved.