NICT’s Neural and Statistical Machine Translation Systems
for the WMT18 News Translation Task
Benjamin Marie
Rui WangAtsushi Fujita
Masao Utiyama
Eiichiro Sumita{bmarie, wangrui, atsushi.fujita, mutiyama, eiichiro.sumita}@nict.go.jp
National Institute of Information and Communications Technology (NICT), Kyoto, Japan

SummaryData

The shared tasks

Bilingual data

• News translation, constrained (only provided data are used)

Language pair

• Estonian-English (et-en), Finnish-English (fi-en), Turkish-English(tr-en), and Chinese-English (zh-en)

et-enfi-en
tr-enzh-en

Key points• No specific linguistic pre-processing#sent. pairs
1.9M
3.1M
207.4k
24.8M• SMT and NMT (transformer) systems• Combination of SMT and NMT through n-best lists reranking
The tesults• Best systems (BLEU) for Estonian-English and Finnish-English

29.4M (Et)
52.9M (Fi)4.4M (Tr)509.9M (Zh)Language#lines#tokens

enetfitrzh

338.7M
146.1M
177.1M
105.0M
130.5M

7.5B
3.6B
3.2B
1.8B
2.3B

• Tokenizer:– Moses: en, et, fi and tr– Jieba: zh• Truecased (en, et, fi and tr)• No data filtering

Incremental Training for NMT

Combination of SMT and NMT

Training on more and better back-translated data

Method (Marie+, 18)i=0sourcemonolingual data

36.0M (En)72.8M (En)
5.1M (En)576.2M (En)Pre-processing

Monolingual data

• Incremental training for NMT#tokens

1. Produce n-best lists with NMT and SMT

bilingualparallel datatargetmonolingual data

2. Compute SMT and NMT features for all the hypotheses
i=i+1
source totarget
NMT

3. Merge the n-best lists

targetto source
NMT

4. Rerank the hypotheses given the feature scores
synthetic
parallel datasynthetic
parallel data

ki=rki-1
sentences

ki=rki-1
sentences

Features• Left-right and right-left NMT forward and backward models

kiback-translated
sentenceskiback-translated
sentences• 4-gram language models• SMT lexical translation probabilities

Experiments with tr-en• Averaged word posterior probabilities24• T-200k-2: r = 2, i0 =200k• Hypothesis length features22• T-100k-1.5: r = 1.5, i0 =100k• T-fixed: no incremental training

BLEU

20

• System flag

18
Tr-En, T-200k-2
Tr-En, T-100k-1.5
Tr-En, T-fixed
Tr-En, R-balanced
En-Tr, T-200k-2
En-Tr, T-100k-1.5
En-Tr, T-fixed
En-Tr, R-balanced

1614• R-balanced: backtr. with RNN

120k

500k

1000k
1500k
2000k
2500k
# of synthetic parallel sentences• Phrase-based forced decoding score (for tr-en only) (Zhang+, 18)• MBR decoding for n-best list (for tr-en only)

3000k

Experiments
Systems

Human evaluation• SMT (#3, #4, #5, #6): Moses (hierarchical for et-en and fi-en)• First rank for all tasks (except for zh-en)• NMT (#1, #2, #6): Marian (transformer model)• Many other systems are also ranked first• Reranker (#2, #6): KB-MIRA• Strong correlation with BLEU

Results#System

et→en

en→et

fi→en

en→fi

tr→en

en→tr

zh→en

en→zh

1.
2.

Moses
Moses NMT-reranked

18.2
20.2

15.1
17.6

15.8
17.5

10.7
12.2

12.1
14.2

8.4
10.1

16.9
19.0

28.0
29.9

3.
4.
5.

Marian single (w/o backtr)
Marian single (w/ backtr)
Marian ensemble (w/ backtr)

22.9
28.6
29.1

18.5
24.0
24.3

17.6
23.1
23.6

13.2
16.8
17.3

20.2
25.2
25.8

12.2
18.0
18.3

23.7
24.7
25.9

33.0
37.2
37.9

6Moses + Marian

30.7

25.2

24.9

18.2

26.9

19.2

26.7

39.7

• Moses (#1) underperforms Marian (#3) by several BLEU points• Small improvements with ensemble decoding (6 models) (#5)

• Moses NMT-reranked (#2) is slightly below Marian (#3) for fi-en• Best results when combining SMT and NMT (#6)

• Large improvements with back-translation (#4) (> +5BLEU)• Best systems (#6) of WMT for et-en and fi-en (BLEU)