IITP English-Hindi Machine Translation System at WAT 2016
Sukanta Sen, Debajyoty Banik, Asif Ekbal, Pushpak Bhattacharyya
Department of Computer Science and Engineering
Indian Institute of Technology Patna, IndiaMethods● Two Hierarchical En-Hi SMT○ First: Without external data
○ Second: With external data (bilingualdictionary)● Source-side reordering: to conform thesyntactic structure of the target.
● Baseline: Phrase-based SMT

IITP Machine Translation Pipeline
BilingualDictionary

Training Set

ModifiedCorpus

Source-side
Reordering

Test Set

ApplyHierarchical
SMTDevelopmentSet

Source-side
Reordering

Source-side Reordering● English → subject-verb-object (SVO)● Reordering example○ English: The president of America visited India in June.
○ Reordered: America of the president June in India visited.
○ Hindi: अमे रका के राप त ने जून म भारत क यातरा क ।Augmenting Bilingual Dictionary● Motivation: It improves word alignment[3]● English-Hindi bilingual mapping → extract
parallel words● Augment parallel words with training corpus

Built SMT systems– Phrase-based(Phr)– Hierarchical model(Hie)– Hierarchical model after reordering
source(HieRe)– Hierarchical model with dictionary andreordering (HieReDict)
BLEU

PhrHieHieRe

HieReDict

11.79

13.18

13.57

13.71

Error Analysis
Source-sideReordering

DecoderOutput● Hindi → subject-object-verb (SOV)● Syntactic reordering of the source helps(Collins et al., 2005; Ramanathan et al.,2008)

Results

Experimental Setup● Training using Moses Toolkit● Stanford parser for parsing source sentences● Source-side reordering using CFILT[4]
pre-ordering tool
● GIZA++ with grow-diag-final-and heuristic forword-alignment● 4-gram language model with modified
Kneser-Ney smoothing using KenLM● Distortion limit 6 (only for baseline
Phrase-based SMT)
● Minimum-Error-Rate-Training (MERT) tuningAn example
Source: the rain and cold wind on Wednesday night made people feel cold.
Hie: बुधवार रात को बरसात और ठं डी हवा ने लोग को ठं ड लग रह थी।(budhavAra rAta ko barasAta aura ThaMDI havA ne logoM ko ThaMDa lagarahI thI.)
Observations → wrong post-position ने (ne)HieRe: बुधवार क रात को बरसात और ठं डी हवा से ठं डक महसूस हु ई।(budhavAra kI rAta ko barasAta aura ThaMDI havA se ThaMDaka mahasUsahuI.)
Observations → 1. Correct post-position से (se)2. लोग (logoM) (Gloss: people) is dropped
HieReDict: बुधवार रात बा रश और ठं डी हवा से लोग को ठं ड लगने लगा।(budhavAra rAta bArisha aura ThaMDI havA se logoM ko ThaMDa lagane lagA.)
Observations → लोग (logoM) (Gloss: people) brought backConclusion
We found that hierarchical SMT model, whenaugmented with bilingual dictionary along withsyntactic reordering of English sentences produced
better translation score.

References
1.
2.
3.
4.

Ananthakrishnan Ramanathan, Jayprasad Hegde, Ritesh M Shah, Pushpak Bhattacharyya, and M Sasikumar. 2008. Simple syntactic and morphological processing can help english-hindi statistical machine translation. In IJCNLP, pages 513–520.
Michael Collins, Philipp Koehn, and Ivona Kučerová. 2005. Clause restructuring for statistical machine translation. In Proceedings of the 43rd annual meeting on association for computational linguistics, pages 531–540. ACL.
http://www.cfilt.iitb.ac.in/ sudha/bilingual_mapping.tar.gz
http://www.cfilt.iitb.ac.in/ moses/download/cfilt_preorder