言語処理学会 第21回年次大会 発表論文集 (2015年3月)

Inﬂating Training Data for Statistical Machine Translation using
Unaligned Monolingual Data
Wei YANG, Zhongwen ZHAO, Yves LEPAGE
Graduate School of Information, Production and Systems, Waseda University{kevinyoogi@akane; zzw890827@fuji}.waseda.jp; yves.lepage@waseda.jp

Abstract
In data-driven machine translation, parallel corpora arean extremely important resource. For language pairs thatinvolve English, there exist many freely available bilingual or multilingual parallel corpora, especially for European languages. To improve the translation quality forless-resourced language pairs, such as Chinese–Japanese,larger and larger aligned training data are needed. The
constitution of large bilingual corpora is not easy forless documented language pairs. In this paper, we show
how to construct a Chinese–Japanese quasi-parallel corpus automatically by using analogical associations basedon a small amount of parallel sentences and a reasonableamount of monolingual data. We perform SMT experiments in Chinese–Japanese and compare a baseline system and a system build by adding the quasi-parallel corpus. On the same test set, the translation quality signiﬁcantly improved over the baseline system.

1

method is that, while direct construction of large bilingual corpora is difﬁcult, monolingual data is relatively
easy to access in large amounts. The languages on which
we will illustrate our proposed method in this paper are
Chinese and Japanese.
The procedures in our experiments are as follows:• Cluster and group monolingual data collected fromthe Web using proportional analogy independently
in both Chinese and Japanese. Such clusters canbe considered as rewriting models for new sentence
generation. We also compute the similarity between
these clusters across the two languages.
• Generate new sentences using these rewriting models starting from seed sentences extracted from themonolingual part of some parallel corpus, and ﬁlter
out dubious sentences.
• Construct the quasi-parallel corpus by assessing thestrength of translation relations between the ﬁltered
newly generated sentences based on the similarity
between the clusters they were generated from andthe translation relations between the seed sentences.

Introduction

Sentence-level aligned parallel corpora i.e., sets of parallel sentences, are an important resource as training data instatistical machine translation (SMT), because the necessary translation knowledge is acquired from these sentential parallel corpora, and because the translation relations
between words or phrases between the source language
and the target language are extracted from parallel sentences. There exist numerous freely available sentencelevel bilingual or multilingual parallel corpora for language pairs that involve English, such as the Europarl
parallel corpus [4]. There also exist some studies on parallel sentence extraction that include Chinese or Japaneseas one of the languages involved. Examples are: parallel sentence mining from Chinese–English bilingual web
pages [8]; extraction of Japanese-English parallel sentences from a noisy parallel corpus [7].
Our proposed method follows a slightly different path.
We propose to automatically construct a bilingual corpus of quasi-parallel sentences based on a small amountof parallel corpus and a reasonable amount of unaligned
monolingual data. A quasi-parallel corpus will containaligned sentence pairs that are translations to each otherto a certain extent. The motivation for our proposed• Perform SMT experiments: compare the baseline
system and a system build by adding the quasiparallel corpus as the additional training data.

2Data Preparation

2.1

Chinese–Japanese Parallel Sentences

The Chinese and Japanese linguistic resources we use inthis paper are the ASPEC-JC (Asian Scientiﬁc Paper Excerpt Corpus-Japanese–Chinese) corpus1 . This corpus isdesigned for Machine Translation and is split as follows:• Training Data: 672,315 sentences;• Development Data: 2,090 sentences;• Test Data: 2,107 sentences.
For the monolingual part of the parallel sentences usedin new sentence generation, we extracted short sentenceswith less than 30 characters in length from the Training
1 http://orchid.kuee.kyoto-u.ac.jp/ASPEC/― 1016 ―Copyright(C) 2015 The Association for Natural Language Processing.
All Rights Reserved.

ご 確認 お願いします

Data, and obtained 103,629 sentences to be used as seed
sentences for new sentence generation in both languages.

:‘Please conﬁrm it.’

ご 確認 ください:‘Conﬁrm it, please.’

2.2

Chinese and Japanese MonolingualSentences

For construction of the analogical clusters, we use Chinese and Japanese monolingual data. We collected Chinese and Japanese monolingual short sentences with lessthan 30 characters in size from the Web using an in-house
Web-crawler, mainly from the following websites: “Yahoo China”, “Yahoo China News”, “douban” for Chineseand “Yahoo! JAPAN”, “Mainichi Japan” for Japanese.
We use 70,000 Chinese and Japanese monolingual data
(after some ﬁltering) in clustering experiments. Although
the number of sentences in both language is the same, thesentences are basically unrelated.

3Constructing Analogical Clusters

3.1

Sentential Analogies

We cluster and gather pairs of sentences in Chinese andJapanese independently using proportional analogy. Proportional analogies establish a structural relationship A :B :: C : D between four objects, A, B, C and D: ‘A is toB as C is to D’. An efﬁcient algorithm for the resolutionof analogical equations between strings of characters has
been proposed in [5].
The algorithm relies on counting numbers of occurrences of characters and computing edit distances (withonly insertion and deletion as edit operations) between
strings of characters. For a detailed presentation of theformula and the algorithm, we refer the reader to [5]. We
call sentential analogy a proportional analogy between
sentences.

確認 しました

3.2:

UnderstandI have con- I have under:::it, please.
ﬁrmed.
stood.

Analogical Clusters

When several sentential analogies involve the same pairsof sentences, they form a series of analogous sentences,and they can be written on a sequence of lines where eachline contains one sentence pair and where any two pairsof sentences form a sentential analogy. We call such asequence of lines an analogical cluster. The following
analogical cluster in Japanese shows three possible sentential analogies. Analogical clusters can be considered
as rewriting models to generate new sentences.
Analogical clusters are constructed from the monolingual data separately in each language. Table 1 summarizes some statistics on the clusters produced.

ご 了承 ください‘Understand it, please.’:‘I have conﬁrmed.’# of different sentences# of clusters

了承 しました‘I have understood it.’

Chinese70,00023,182

Japanese
70,00021,975

Table 1: Statistics on the Chinese and Japanese clusters
constructed independently in each language.

3.3

Determining corresponding clusters bycomputing similarity

For determining corresponding clusters, we consider thechanges between the left (Sle f t ) and the right (Sright ) sides
in one cluster as two sets. We perform word segmentation2 on these changes in sets to obtain minimal sets ofchanges made up with words or characters.
Then, we compute the similarity between the left sets
(Sle f t ) and the right sets (Sright ) of Chinese and Japaneseclusters. To this end, we make use of the EDR dictionary3
and word-to-word alignments (based on ASPEC-JC datausing Anymalign4 ), we keep 72,610 word-to-word correspondences obtained with Anymalign in 1 hour afterﬁltering on both translation probabilities with a thresholdof 0.3 (approximate correctness judged by hand: 95%).
We also use a traditional-simpliﬁed Chinese variant table5 and Kanji-Hanzi Conversion Table6 to translate all
Japanese words into Chinese, or convert Japanese characters into simpliﬁed Chinese characters. We calculate
the similarity between two Chinese and Japanese word
sets according to a classical Dice formula:Sim =ご確認く ご 了 承 く 確 認 し 了承しま::::ださい
ださい
ました したConﬁrm it, please.

ご 了承 お願いします‘Please understand it.’

2 × |Szh ∩ S ja ||Szh | + |S ja |(1)

Szh and S ja denote the minimal sets of changes across theclusters (both on the left or right) in both languages (aftertranslation and conversion). To compute the similarity
between two Chinese and Japanese clusters we take thearithmetic mean on both sides, as given in formula (2):1
(2)
SimCzh −C ja = (Simle f t + Simright )2About 15,710 cluster pairs were judged valid corresponding clusters (SimCzh −Cja ≥ 0.300) by the above
2 Segmentation toolkits: Mecab: http://mecab.googlecode.
com/svn/trunk/mecab/doc/index.html for Japanese and Urheen,a Chinese lexical analysis toolkit (National Laboratory of Pattern
Recognition, China) for Chinese.
3 http://www2.nict.go.jp/out-promotion/techtransfer/EDR/index.html
4 http://anymalign.limsi.fr
5 http://www.unicode.org/Public/UNIDATA/
6 http://www.kishugiken.co.jp/cn/code10d.html― 1017 ―Copyright(C) 2015 The Association for Natural Language Processing.
All Rights Reserved.

Generation of New Sentences

Analogy is not only a structural relationship. It is also aprocess [3] by which, “given two related forms and only
one form, the fourth missing form is coined” [1]. If theobjects A, B, C are given, we may obtain another object
D according to the analogical equation A : B :: C : D. It
can be illustrated with sentences:
あらかじめご確認お願い ご 了 承 お 願::: ご 確 認 く だ : x
します
いしますさい⇒ x = あらかじめご了承ください
Here, the solution of the analogical equation is D =“あらかじめご了承ください” (Please understand itpreviously.). If we regard each sentence pair in a cluster
as a pair A : B (left to right or right to left), and any
short sentence not belonging to the cluster as C (a seedsentence), a new sentence D can be forged.

4.2

Experiments on New Sentence Generation and Filtering

For the generation of new sentences, we made use of theclusters obtained in Section 3.2. The seed sentences are
the monolingual part of Chinese and Japanese short sentences from the 103,629 ASPEC-JC parallel data. In thisexperiment, we generated new sentences with each pairof sentences in clusters for Chinese and Japanese respectively. We obtained about 105 million newly generated
sentences for Chinese and about 80 million for Japanese.
The grammatical quality, as assessed manually, is only of
29% and 40% of the sentences.
To ﬁlter out invalid and grammatically incorrect sentences and keep only well-formed sentences, we eliminate any sentence that contains an N-sequence of a given
length unseen in the reference corpus. Similar techniques have been previously used [2]. In our experiment, we introduced begin/end markers to make sure thatthe beginning and the end of a sentence are also correct. The best quality was obtained for the values N=6for Chinese and N=7 for Japanese on our reference corpora (about 1,700,000 sentences for both Chinese andJapanese). Quality assessment was performed on a sample of 1,000 sentences randomly and manually check bynative speakers. The grammatical quality was at least
96%. For new valid sentences, we remember their corresponding seed sentence and the cluster they were generated from.

5SMT Experiments

5.1

Baseline Training Data = ASPEC-JC,
seeds = short sentences in ASPEC-JC

To assess the contribution of the generated quasi-parallelcorpus, we propose to compare two SMT systems. Theﬁrst one is trained using the ASPEC-JC parallel corpus(baseline). The second one is trained on the ASPEC-JC
data plus the additional quasi-parallel corpus. The segmentation tools used are Urheen for Chinese and Mecabfor Japanese.

train

4.1

We deduce translation relations based on the short parallel sentences for new sentence generation, and the correspondence between clusters in Chinese and Japanese.
If the seeds of two new generated sentences in Chineseand Japanese are aligned in the short parallel sentences,and if the clusters which they were generated from arecorresponding, we suppose that these two Chinese andJapanese newly generated sentences are translations of
one another to a certain extent. We obtained 35,817
unique Chinese–Japanese aligned sentences. Amongthese aligned sentences, about 74% were found to be exact translations by manual check on a sampling of 1,000
pairs of sentences. This ﬁgure of 74% justify our useof the word quasi-parallel to characterize this corpus ofaligned sentences.

train

Generating New Sentences Using
Analogical Associations

Deducing and Acquiring Quasi-parallel
Sentences

tune

4

4.3

test

steps. It is important to stress that 1/ the number of lines
in corresponding clusters may be different, and 2/ thesentences in corresponding clusters are not translations
in general, only the changes are in correspondence.

Baseline
sentenceswords
mean ± std.dev.
+ Quasi-parallel
sentenceswords
mean ± std.dev.

Chinese672,315
18,847,514
28.12 ± 15.20
Chinese708,132
19,175,105
27.61 ± 15.33Japanese
672,315
23,480,703
35.05 ± 18.88
Japanese
708,132
23,867,354
34.47 ± 19.07

Both experiments
sentenceswords
mean ± std.dev.
sentenceswords
mean ± std.dev.

Chinese2,090
60,45828.93 ± 15.86
2,107
59,59428.28 ± 14.55

Japanese
2,090
73,17735.01 ± 18.87
2,107
72,02734.18 ± 17.43

Table 2: Statistics when baseline training data = all sentences in ASPEC-JC.

We perform all experiments using the standard
GIZA++/MOSES 1.0 pipeline [6]. As Table 3 shows,signiﬁcant improvement over the baseline is obtained byadding the quasi-parallel generated data.
― 1018 ―Copyright(C) 2015 The Association for Natural Language Processing.
All Rights Reserved.

zh-jaja-zhbaseline+ additional training databaseline+ additional training data

BLEU
29.10
32.03
22.98
24.87

Table 3: Evaluation results when baseline training data =all sentences in ASPEC-JC. Boldfaced numbers stand forstatistically signiﬁcant improvements.

5.2

Baseline Training Data = seeds = short
sentences in ASPEC-JC

test

tune

train

train

In a second experiment, the training data is the same as
the data used as seeds in sentence generation, i.e., weeliminated from the training data sentences with a length
greater than 30 characters. Similarly, we inspect the impact of adding the quasi-parallel corpus.
Baseline
sentenceswords
mean ± std.dev.
+ Quasi-parallel
sentenceswords
mean ± std.dev.

Chinese103,629
1,147,636
11.19 ± 3.58
Chinese139,446
1,475,227
10.82 ± 3.74

Japanese
103,629
1,407,873
13.73 ± 4.12
Japanese
139,446
1,794,524
13.13 ± 4.35

Both experiments
sentenceswords
mean ± std.dev.
sentenceswords
mean ± std.dev.

Chinese2,090
60,45828.93 ± 15.86
2,107
59,59428.28 ± 14.55

Japanese
2,090
73,17735.01 ± 18.87
2,107
72,02734.18 ± 17.43

Table 4: Statistics when baseline training data = short
sentences in ASPEC-JC.
The experimental results are shown in Table 5.

zh-ja
ja-zh

baseline+ additional training databaseline+ additional training data

BLEU
18.49
24.30
15.89
19.88

Table 5: Evaluation results when baseline training data =short sentences in ASPEC-JC. Boldfaced numbers standfor statistically signiﬁcant improvements.

6ConclusionsIn this paper, we presented a different way to inﬂate
the training corpus used to improve the translation quality for the SMT system: we automatically generate aquasi-parallel corpus. The experimental data we use are
ASPEC-JC corpus and certain amount of monolingual

data collected from the Web. We use analogical associations to generate new sentences, and ﬁlter them to ensure
grammatical correctness. The grammatical quality of thevalid new sentences is at least 96%. We then assess translation relations between valid newly generated sentences
across both languages, relying on the similarity between
the clusters across languages. We automatically obtained
a quasi-parallel corpus of 35,817 Chinese–Japanese sentences, 74% of which were found to be exact translations.
In SMT experiments, we inﬂated the training data with
our quasi-parallel data in a rewarding way. On the sametest set, the translation scores signiﬁcantly improved overthe baseline systems. It should be stressed that the datathat allowed us to get such improvement are not so large
in quantity and not so good in quality, but we were able
to control both quantity and quality so as to consistently
improve translation quality.

References[1] Ferdinand de Saussure.
Cours de linguistiquegénérale. Payot, Lausanne et Paris, [1ère éd. 1916]
edition, 1995.
[2] George Doddington. Automatic evaluation of machine translation quality using n-gram co-occurrence
statistics. In Proceedings of the Human Language
Technology Conference (HLT2002), pages 128–132,
San Diego, CA, USA, 2002. Morgan Kaufmann.
[3] Esa Itkonen. Analogy as Structure and Process:Approaches in linguistics, cognitive psychology andphilosophy of science, volume 14.
John Benjamins Publishing Company, Amsterdam / Philadelphia, 2005.
[4] Philipp Koehn. Europarl: A parallel corpus for statistical machine translation. In Proceedings of the tenth
Machine Translation Summit (MT Summit X), pages
79–86, Phuket, Thailand, 2005.
[5] Yves Lepage and Etienne Denoual. Purest everexample-based machine translation: detailed presentation and assessment. Machine Translation, 19:251–282, 2005.
[6] Franz Josef Och and Hermann Ney. A systematiccomparison of various statistical alignment models.
Computational Linguistics, 29(1):19–51, 2003.
[7] Masao Utiyama and Hitoshi Isahara. Reliable measures for aligning japanese-english news articles andsentences. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics,pages 72–79, 2003.
[8] Zhenxiang Yan, Yanhui Feng, Yu Hong, and Jianmin Yao. Parallel sentences mining from the web.
Journal of Computational Information Systems, 5(6):1633–1641, 2009.
― 1019 ―Copyright(C) 2015 The Association for Natural Language Processing.
All Rights Reserved.