Translation systems andexperimental resultsof the EHR groupfor WAT2016 tasks
Terumasa EHARA
Ehara NLP Research Laboratory
http://www.ne.jp/asahi/eharate/eharate/
1

Participated tasks andused techniques
Tasken-ja
zh-ja
JPCzh-ja
JPCko-ja
HINDENen-hi
HINDENhi-ja

CharacterWord-based
based
RBMT+SPE Reordering Pivoting
PBSMTPBSMT✔
✔
✔
✔
✔
✔
✔
✔
✔
✔
✔
✔
✔
✔
✔
✔

PBSMT : Moses V.3, MGIZA++ v. 0.7.0
RBMT : Commercial system
SPE : Statistical post editing by Moses
2

PBSMT setting• TM training data filtering out> 100 words
ratio of word numbers is > 4 or < 0.25
• TM training and decoding
Moses V.3, MGIZA++ v. 0.7.0• LM training lmplz order=6• Distortion limit
0 (JPCko-ja task)
6 (other tasks)
3

Reordering system (training data)
Input sentenceN-best parse (100)Reordering system
K-best reordered sentences
Alignment
Reranking (Kendall’s tau)

ParserBerkeley parser
Original rule for en
Tuned rule for zh(Stanford parser)
Reordering system
Rule based.
Deletion of articles (a, an and the) andadding case markers (subj and obj) for
en-ja and en-hi tasks.
Deletion of case markers (が, は, を) for
zh-ja task.
Aligner
MGIZA++
Self-built post processor for GIZA++
outputs (A3.finals, lex.f2e and lex.e2f)

Output top ranked reordered sentence4Reordering system (dev devtestand test data)
ParserInput sentenceN-best parse (100)Reordering system
K-best reordered sentences
Reranking (LM score)

Berkeley parser
Original rule for en
Tuned rule for zh(Stanford parser)
Reordering system
Rule based.
Deletion of articles (a, an and the) andadding case markers (subj and obj) for
en-ja and en-hi tasks.
Deletion of case markers (が, は, を) forzh-ja task.
Reranking
Query command in Moses

Output top ranked reordered sentence5

en-ja task and HINDENen-hi task
• Moses tokenizer for en• Indic NLP normalizer and tokenizer for hi• JUMAN for ja• Training corpus size
TM 1,502,767 (en-ja) alignment score≧0.08
1,450,896 (en-hi) filter out 21,637 data
LM 3,824,408 (en-ja) from en-ja and hi-ja tasks
1,599,708 (en-hi) from en-hi and ja-hi tasks
6zh-ja task and JPCzh-ja task
• Stanford Chinese segmenterplus self-built post processor for zh• JUMAN plus self-built post processor for ja• RBMT+SPE for JPCzh-ja task
• Character base only for zh side
• Merging of three outputs (word based SMT,character based SMT and RBMT+SPE) by LM score7zh-ja task and JPCzh-ja task
• Training corpus size
TM 667,922 (zh-ja) zh-ja task data
995,385 (JPCzh-ja) JPCzh-ja task data
LM 3,680,815 (zh-ja) from zh-ja anden-ja task data
4,186,284 (JPCzh-ja) from JPCzh-ja task and
NTCIR-10’s en-ja task data8

JPCko-ja task
Mecab-ko for ko tokenizer
JUMAN for ja segmenter
Character base both for ko and ja side
Merging of two outputs (word based SMT andcharacter based SMT) by LM score• Handling of parentheses surrounding a number :(1) delete paren. to ko side(2) add paren. to ja side(3) add paren. to ja side and delete them after
decoding••••9

JPCko-ja task
• Training corpus size
TM 996,339 JPCko-ja task data
LM 5,186,284 from JPCko-ja, JPCzh-ja
and NTCIR-10’s en-ja task data

10

HINDENhi-ja task
• Four methods is conducted :(1) Simple PBSMT (direct translation)(2) Sentence level pivoting without reordering(3) Sentence level pivoting with reordering(4) Table level pivoting with reordering• User dictionary : 931 words (OOV in dev and test)• Training corpus size for the method (1)TM 149,743 hi-ja task data plus user dictionary
LM 406,766 hi-ja task data plus TED corpus
11

Sentence level pivoting
1,450,896 (HINDEN hi-en)
hihien( Reordering )1,450,896

149,743

en-ja SMTjaja
Direct hi-ja corpus

Pivoted corpus

Total TM training corpus
1,600,639
12

Table level pivoting
1,450,896 (HINDEN hi-en)
Moses

hi

hi-ja phrase table(pivoted)enhi-en phrase tablehi-en reordering table

Merging

Merging

en-ja phrase tableen-ja reordering tablehi-ja reordering table(pivoted)

Merging

Mergingenhi-ja phrase table(direct)

Mosesja
hi-ja reordering table(direct)

1,502,767 (ASPEC en-ja)

hiMoses
149,743 (HINDEN hi-ja)

ja
13

Phrase table pivoting∅( | ) =( | )=∅( | ) =( | )=∅( | ) ∅ ( | )
( | )

( | )

∅( | ) ∅ ( | )
( | )

( | )

filter out for∅( | )∅( | ) < 0.000001

f : source (hi)e : target (ja)p : pivot (en)14

Phrase table merging∅ ( | )F ( )+∅ ( | )F ( )∅( | ) =.
F ( )+ F ( )p : pivoted
d : direct
Fp : frequency in the pivoted corpus
Fd : frequency in the direct corpus

15

Pivoted reordering table orientation
fp＼pemsdmmsdss
msddsmfp : source (hi) to pivot (en) orientationpe : pivot (en) to target (ja) orientationm : monotones : swapd : discontinuous
16Reordering table pivotingm (f → e) ={m(f → p)m(p → e) + s(f → p)s(p → e)

+ d(f → p)d(p → e)}/s(f → e) ={m(f → p)s(p → e) + s(f → p)m(p → e)+ d(f → p)s(p → e) + s(f → p)d(p → e)} /d(f → e) ={m(f → p)d(p → e) + d(f → p)m(p → e)} /D

D : normalizer such that m f → e + s f → e + d f → e = 1
17

Reordering table merging
a (f → e) F (f) + a (f → e) F (f)a(f → e) =F (f) + F (f)a : {m|s|d}p : pivoted
d : direct
Fp : frequency in the pivoted corpus
Fd : frequency in the direct corpus

18

Results of iterative reordering(JPCzh-ja)
Kendall's tau

0.87900.87850.87800.87750.8770
123

4

5678

Iteration
19

Results of iterative reordering
Task
Iteration Kendall's tau
en-ja
40.7655
zh-ja
40.9083
JPCzh-ja
80.8788
HINDENen-hi
40.839820

Evaluation result of system combination(JPCzh-ja)
No.

System
1 word based SMT
2 char based SMT
3 RBMT + SPE
4 to combine 1 and 2
5 to combine 1, 2 and 3

BLEU
RIBES
42.07
82.91
41.82
83.03
41.61
82.42
42.13
83.13
42.4283.16

21

Evaluated systems
Task
en-ja
zh-ja
JPCzh-ja
JPCko-ja
HINDENen-hi
HINDENhi-ja

CharacterSentence
System Word-based
based
RBMT+SPE Reordering level
No.
PBSMT
PBSMT
pivoting
1
✔
✔
1
✔
✔
✔
1
✔
✔
✔
✔
2
✔
✔
✔
1
✔
✔
2
✔
✔
3
✔
✔
1
✔
✔
1
✔
✔
2
✔
✔
✔
3
✔
✔
4
✔

Table level Parenthes
pivoting
handlingdeladd & deladd✔

22

HINDENhi-ja

0.75990.84370.82700.82800.9447
0.94110.92710.67190.57930.58600.58230.44020.74670.76950.73500.74510.86640.85170.81800.65080.46810.47310.45490.362839.00032.50035.50039.000-3.000--21.750
0.000
13.75010.000-----55.25063.75046.50021.75057.25039.750

JPO adq.
(top team)

31.32
39.75
41.05
40.95
71.51
68.78
62.33
11.75
7.81
7.66
7.47
2.36

JPO adq.

HUMAN(top team)

HINDENen-hi

HUMAN

JPCko-ja

AMFM

JPCzh-ja

1
1
1
2
1
2
3
1
1
2
3
4

RIBES

en-ja
zh-ja

BLEU

Task

System No.

Evaluation results------------4.562.48
2.00-------

4.02
3.94
3.44
4.62
2.55
2.13

23

(%)

Evaluation results by JPO adequacy
1009080
70
160

250

3440

530
20
100
JPCko-ja

HINDENen-hi

HINDENhi-ja

24Conclusion• Our translation techniques are effective
Iterative reordering
System combination
Pivoting with reordering• Remaining issues
To improve parsing accuracy
To improve hi-ja and en-hi accuracy
To challenge MT for other Asian languages(Indonesian, Thai, Vietnamese, Mongolian, etc.)25