The NAIST Machine Translation System for IWSLT 2012

The NAIST Machine Translation
System for IWSLT 2012

Graham Neubig, Kevin Duh, Masaya Ogushi, Takamoto Kano
Tetsuo Kiso, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura
Nara Institute of Science and Technology
12/6/20121

The NAIST Machine Translation System for IWSLT 2012

Overview●

Phrase-based machine translation●Built on Moses (experiment management system)●

Evaluated on TED Translation:
English → French official track● XXX → English other tracks●Focus:easily implementablelanguage-independentmethods

2The NAIST Machine Translation System for IWSLT 2012English-French

3

The NAIST Machine Translation System for IWSLT 2012

Summary of English-French●Four successful statistical methods:Phrase-table smoothing● Language model interpolation● Calibrated minimum Bayes risk decoding● Large-scale data with filtering●●

Combination raises BLEU 29.75 → 31.81●

Ablation tests to examine the factors

4

The NAIST Machine Translation System for IWSLT 2012

Phrase Table Smoothing●●●

Phrase probabilities for rare phrases over-fit thetraining dataSmoothing discounts observed counts whencalculating probabilities
Here we test Good-Turing smoothing [Foster 06]

5

The NAIST Machine Translation System for IWSLT 2012BLEU

Smoothing Results
333231302928272625

No SmoothingGood-Turing+0.62+0.30

dev2010

tst2010

6

The NAIST Machine Translation System for IWSLT 2012

Language Model Interpolation●LM data from heterogeneous sources●TED, News Commentary, Europarl, Giga

●Combine using simple linear interpolation●Maximize likelihood of development set [Jelinek 80]

7

The NAIST Machine Translation System for IWSLT 2012

Language Model Interpolation Results
33BLEU

312927TED Only
Without Interp
With Interp+1.50+1.71+2.37+2.25

25
23

dev2010

tst2010

8The NAIST Machine Translation System for IWSLT 2012

Minimum Bayes Risk Decoding●●Normal Decoding: Choose the translation with highest
probability
MBR Decoding: From an n-best list, choose thetranslation with the lowest expected loss [Kumar 04]●

Lattice MBR Decoding: MBR over lattices [Tromble 08]●Also tested calibrating the probability distribution

9

The NAIST Machine Translation System for IWSLT 2012BLEU

Minimum Bayes Risk Results
333231302928272625

No MBRMBR
Lattice MBR+ Calibration-0.30+0.80+0.23 +0.24-0.89 -0.54

dev2010

tst2010
10The NAIST Machine Translation System for IWSLT 2012

Large-Scale Data with Filtering●Giga-word English-French corpus is large, but noisy●Train a classifier to detect noisy sentences●●Features: Model 1, Alignment, Length Ratio, Same Word

Use pseudo-negative training examples by swapping
30% of sentences [Mediani 2011]

F1F2F3F4E1E2E3E4F1F3F2F4E1E2E3E4+1-1-1+111The NAIST Machine Translation System for IWSLT 2012

BLEU

Data Filtering Results
333231302928272625

TED+NC+EP+Unfiltered GIGA+Filtered GIGA+0.38 +0.29+0.42 +0.44

dev2010

tst2010

12

The NAIST Machine Translation System for IWSLT 2012

Other Methods Investigated●Out of domain TM data●

Word alignment methods + combination●Lexical reordering models●MERT vs. PRO tuning

See the paper for more details!

13

The NAIST Machine Translation System for IWSLT 2012

XXX-English Language Pairs

14

The NAIST Machine Translation System for IWSLT 2012Linguistic Family Tree
●Indo-European Family:
Germantic: German (de), Dutch (nl), English (en)● Italic: Portuguese (pt), Romanian (ro)● Slavic: Polish (pl), Russian (ru), Slovak (sk)●●Afro-Asiatic Family: Arabic (ar)●Altaic Family: Turkish (tr)15

The NAIST Machine Translation System for IWSLT 2012

MT Issues●Morphology:
pl/ru/sk (fusional)● tr (agglutinative)● de/nl (compounding)● pt/ro (some inflection)●●Word order:
de/nl (SOV, V2)● ar (VSO)●16

The NAIST Machine Translation System for IWSLT 2012

Summary of XXX-English Systems●●Common EMS setup: compare performance ofexisting techniques cross-linguistically
What worked generally:●

Unsupervised Morphology–●

Using Morfessor and compound-splitter.perl

Gigaword LM17

The NAIST Machine Translation System for IWSLT 2012

Unsupervised Morphology●

Compound-splitter.perl [Koehn 03]●●

Breaks apart words if subparts are seen in training dataover a certain frequency

Morfessor 1.0 [Creutz 02]
Use Minimum Description Length principle to find asmall set of morphemes that covers the training words● Discovers both free & bound morphemes● Small modification: Morfessor segments too
aggressively for unknown words, so keep OOV as is●18

The NAIST Machine Translation System for IWSLT 2012

Vocabulary Growth Rateardenlplpt

rorusktr

19

The NAIST Machine Translation System for IWSLT 2012

Morphology Results
2Gain in BLEU

1.5

Compound Split
Morfessor+1.5+1.0

1+0.5

0.5+0.20

ardenl+0.3+0.2

plptrorusktr-0.5
20

The NAIST Machine Translation System for IWSLT 2012

Language Model Addition
Added additional Giga-Word language model
0.8
0.6

Gain in BLEU

●

0.4
0.2
0-0.2

ardenlplptrorusktr-0.4
21

The NAIST Machine Translation System for IWSLT 2012

Other Methods Investigated●Out of domain TM data

●Lattice-based MBR

See the paper for more details!

22

The NAIST Machine Translation System for IWSLT 2012

Thank You!

23