NICT-2 Translation System for WAT2016:
Applying Domain Adaptation to Phrase-based Statistical Machine Translation
Summary
WAT2016 data.

Japan Patent Office Corpus (JPC) was regarded as a mixture offour domain corpora.
 Domain adaptation was effective on the patent data
even if the domains are different.
 We added ASPEC as the fifth domain, but there were no effects.
 The patent data was not effective to the scientific paper domain.
 Google n-gram language models are added as external knowledge.
Experimental Results

Domain/Corpora Domain adaptation method of Imamura+ (2016) was applied toOur domain adaptation can easily incorporate such knowledge.

Domain Adaptation (Imamura+ 2016)
Adaptation of Weight Vector

 Japan Patent Office Corpus (JPC) was regarded as a mixture of fourdomains.

 Asian Scientific Paper Excerpt Corpus (ASPEC) was used as the fifthdomain corpus.
 The language pairs: Japanese-English (Ja-En) and JapaneseChinese (Ja-Zh).
Corpus
DomainJPC
Chemistry
Electricity
MachinePhysics
ASPEC
ASPEC

#training sents.
Ja-En pair
Ja-Zh pair
250k
250k
250k
250k
250k
250k
250k
250k
1,000k672k

 Feature weights are optimized using feature augmentation(Daumé 2007).
 A feature space is expanded to common and domain-specific spaces.
 All domains are simultaneously optimized/adapted.
CommonFeature Space
Domain 1 Domain 2

Implementation Notices

Domain 1 Data

corpus-concatenated or single domain models (unknown probability).
Common
Domain i
Φ translation, 翻訳 Φ translation, 翻訳

Domain 2 Data(-1.7, -6.3, -2.2, -7.6)
Domain D Data
Optimization
Feature Weights
Subspaces used forthe translation in the domain 1

Subspaces used forthe translation in the domain 2

Adaptation of Feature Vector
 Models are changed according to the feature spaces.

For the common space,
Domainwe use a corpus-concatenated model,
1which is trained from corpora of all domains.
DomainDomain
 For the domain specific spaces,
2
3
we use single-domain models,
which are trained from specific domain data.
 This distinction matches meanings of the spaces.

Decoding Procedure
1. Phrase pairs are retrieved from both the corpus-concatenated andsingle-domain phrase tables.
2. Features of the corpus-concatenated model are located to thecommon space, and those of the single-domain model are located tothe domain-specific space.
3. During search of the best hypothesis, the likelihoods are computed
using only the common space and domain-specific space of theinput sentence.

Settings
 Domain Adaptation vs. {Single-Domain / Corpus Concatenation}

Evaluation Metric: BLEU
Statistical Testing: MultEval (p<0.05).
 The scores are different from the official scores.
JPO Corpus (w/o External Knowledge)
Corpus Concatenation:(BLEU)
JPC wasregarded as one domain corpus.
を、単独ドメインモデル、コーパス結合モデ ドメイン適応の翻訳品質
Single Domain Model: If we divided JPC into 4 domains,
 ルと比較する形で測定

the有意差検定のため、
translation qualityMultEval
decreasedbecause thenumber of the training
ツールを使用(p<0.05)sentencesin eachdomain is reduced.
Official scoreとは若干異なる。

 Domain Adaptation: The BLEU scores were the highest.

JPC
Method
Ja-En
En-Ja
Ja-Zh
Zh-Ja
Corpus Concatenation 36.22
38.03 (-) 32.92 (-) 39.68 (-)
Single-Domain Model 35.12 (-) 37.40 (-) 31.96 (-) 38.15 (-)
Domain Adaptation
36.29
38.48
33.36
39.85

Empty Value A value of feature functions when phrases appear only one of theDomain DKenji Imamura and Eiichiro Sumita
National Institute of Information and Communications Technology?? if the pair does notexist in the phrase table.

 We experimentally set to maximize the BLEU score of thedevelopment set.
 This time, empty= -7 (i.e., exp(-7) = 0.0009).

Large Monolingual Corpora
 External knowledge such as language models constructed from largemonolingual corpora is located to the common space while increasing
the dimension.

 Language models are constructed from Google n-gram, and added as

the external knowledge.
 The back-off models are estimated using maximum likelihood.
 English Data : Web 1T 5-gram Version 1 (LDC2006T13)
Japanese Data: Web Japanese N-gram Version 1(http://www.gsk.or.jp/catalog/gsk2007-c/)

Optimization
 Independent optimization of Imamura+ (2016) was used.
 Each domain is optimized one-by-one.
 Optimization algorithm: K-best Batch MIRA.
Translation System Phrase-based SMT with preordering.
 Two preorderers:(1) Top-Down BTG (w/o external knowledge), and(2) In-house preorderer tuned to patents(w/ external knowledge, using Berkeley Parser).
 Moses clone decoder.

JPO and ASPEC Corpus (w/ External Knowledge)
を、単独ドメインモデル、コーパス結合モデ
On JPC, Google n-gram(BLEU)
the language
models and domain adaptation
 ドメイン適応の翻訳品質

ルと比較する形で測定
were both effective. They can be combined.
MultEvalツールを使用(p<0.05)
On有意差検定のため、
ASPEC, domain adaptation
was not effective.
 とは若干異なる。
Official
Thismightscore
be because
the corpus size of ASPEC is large.
JPCMethod
Ja-En
En-Ja
Ja-Zh
Zh-Ja
w/o Corpus Concatenation 35.81 (-) 38.62 (-) 32.76 (-) 39.96 (-)
GN Single-Domain Model 33.90 (-) 38.19 (-) 31.78 (-) 38.74 (-)
Domain Adaptation
36.25
39.58
33.53
40.76
w/ Corpus Concatenation 36.03 (-) 39.48 (-)
40.14 (-)
GN Single-Domain Model 34.35 (-) 39.04 (-)
38.90 (-)
Domain Adaptation
36.40
40.32
40.77
ASPEC
Method
Ja-En
En-Ja
Ja-Zh
Zh-Ja
w/o Corpus Concatenation 22.20 (-) 33.94 (-) 28.95 (-)
37.62 (-)
GN Single-Domain Model
22.79
34.80
29.47 (+) 38.96 (-)
Domain Adaptation
22.80
34.91
29.28
39.18
w/ Corpus Concatenation 22.10 (-) 34.55 (-)
38.15 (-)
GN Single-Domain Model 22.87 (+) 35.42
39.74 (-)
Domain Adaptation
22.74
35.36
39.87

References Hal Daumé III. 2007. Frustratingly Easy Domain Adaptation.

In Proc. of ACL-2007, pp. 256-263.

 Kenji Imamura and Eiichiro Sumita. 2016. Multi-domain Adaptation for

Statistical Machine Translation Based on Feature Augmentation.
In Proc. o AMTA-2016. pp. 79-92.