The POSTECH
Statistical Machine Translation Systemsfor NTCIR-7 Patent Translation Task
Jin-Ji Li0, Hwi-Dong Na0, Hankyong Kim*, Chang-Hu Jin*, and Jong-Hyeok Lee0
0Department of Computer Science and Engineering,Electrical and Computer Engineering Division,*Graduate School for Information Technology,Pohang University of Science and Technology (POSTECH),
San 31 Hyoja Dong, Pohang, 790-784, R. of Korea
E-mail: {ljj, leona, arch, hchchh, jhlee}@postech.ac.kr

Contents[2]

Introduction
Japanese-to-English phrase-based SMT
Reordering model as preprocessing
Cluster-based model as post-processing

ExperimentsConclusion & Future work

Introduction[3]

State-of-the-art system: Phrase-based SMT
However, [Kevin Knight, 2007] said
Translation output is ‘n-grammatical’, not grammatical
Re-ordering is poorly explained as ‘distortion’

Linguistically-distant language pairs require moresophisticated linguistic knowledge
Japanese and English
Big differences in morphological & word-order typologies

How to effectively encode linguistic knowledge intoSMT?

Where & How to Encode Linguistic Knowledge?
[4]

In what steps of SMT system to apply?
Preprocessing
Interlingua

Source word reordering modelPost-processing
Cluster-based model

Semantics/Deep Syntax

Semantics/Deep Syntax

LinguisticallySyntaxLinguisticallySyntaxFormally
SyntaxPhraseWord

Formally
SyntaxPhrase-based SMT+ linguistic knowledge

PhraseWord

Word Reordering Model as Preprocessing[5]Motivation
Japanese and English is linguistically distant language pairSOV vs. SVO language

Reorder the word sequence of source language similar totarget language before decoding
A complement to a phrase-based SMT system which uses a
relatively simple distortion model in the decoding phase

System overviewParse Japanese sentences into dependency trees
Apply a series of manually constructed reordering rules toeach node recursively
Recover the surface strings from the reconstructed
dependency trees

Word Reordering Model as Preprocessing[6]Ex) Reordering rules……

……

Word Reordering Model as Preprocessing
[7]

Ex)

Cluster-based Model[8]

Motivation
Usually, sentences with similar syntactic structures yield
similar distributions of n-grams reflecting their word order
Cluster-specific LM benefits SMT system

System overview
1. Predict clusters according to cluster types
2. Translate using baseline SMT system
3. Optimize LM integration parameters
4. Re-translate using general + cluster-specific LM
5. Select best translation result

Cluster-based Model[9]27 cluster types of syntactic patterns
Subtree structures in the source dependency trees which ignore
the adjuncts as cluster types

Ex) Cluster type list……

……

NTCIR Corpus Profile[10]
Training corpus (1,172,709 sentences)
ChineseKoreanNumber of words

30,761,076

28,683,697

Number of singletons

131,219

131,321

Average length

26.23

24.46

Development corpus (609 sentences)
JapaneseKoreanNumber of words

15,997

14,818

Number of singletons

2,697

2,817

Average length

26.27

24.33

Test corpus (1,381 sentences)
Japanese

EnglishNumber of words

48,27844,910

Number of singletons

4,088

4,273

Average length

34.96

32.52

Experimental Scenario[11]Corpus processing
Japanese: Cabocha tokenizer and parser
http://chasen.org/~taku/software/cabocha/

English-to-Japanese SMT
Vanilla MOSES with 5-gram LM

Japanese-to-English SMT
Vanilla MOSES with syntactic motivated knowledge
Source word reordering model as preprocessing
Cluster-based model as post-processing

NTCIR7 Results[12]1239This is not the official experimental results.

Conclusion & Future Work
[13]

Source word reordering model
Need human evaluation to verity the effectiveness ofproposed method

Cluster-based model
Applied to a small size corpus, it worked better than whenapplied to a large size one
Cluster types are too simple which can cause multiple
matching

Enlightening SMT with various linguistic knowledge
Developing more elaborate reordering rules & applying
other cluster types

Reference[14][1] Michael Collins, Philip Koehn, and Ivona Kučerová, Clause restructuring for statisticalmachine translation, In Proceedings of the 43rd Annual Meeting on Association for
Computational Linguistics, 2005
[2] Fei Xia and Michael McCord, Improving a statistical MT system with automaticallylearned rewrite patterns. In Proceedings of the 20th international Conference on
Computational Linguistics, 2004
[3] Chao Wang, Michael Collins, Philip Koehn, Chinese Syntactic Reordering for Statistical
Machine Translation Proceedings of the 2007 Joint Conference on Empirical Methods in
Natural Language Processing and Computational Natural Language Learning, pp.737-745,
2007[4] Deepa Gupta, Mauro Cettolo, and Marcello Federico. POS-based reordering models forstatistical machine translation. In Proceedings of Machine Translation Summit XI, pp. 207213, 2007
[5] Kay Rottmann and Stephan Vogel. Word reordering in statistical machine translation
with a POS-based distortion model; Proceedings of the 11th International Conference on
Theoretical and Methodological Issues in Machine Translation, 2007
[6]
Yuqi Zhang,RichardZens,andHermann
Ney:
Chunk-level
reordering
ofsource[6]
YuqiZhang,RichardZens,andHermann
Ney:
Chunk-level
reorderingofsource atsentences with automatically learned rules for statistical machine translation. Workshop
op at NAACL-HLT 2007 “Syntax and structure in statistical translation”, pp.1-8, 2007

Reference[15][7] Yuqi Zhang, Richard Zens, Hermann Ney: Improved chunk-level reordering forstatistical machine translation. In Proceeding of International Workshop on Spoken
Language Translation, 2007
[8] Sasa Hasan, Hermann Ney. Clustered Language Models based on Regular Expressions forSMT, 2005, EAMT
[9] Hirofumi Yamamoto, Eiichiro Sumita. Bilingual Cluster Based Models for Statistical
Machine Translation, 2007, EMNLP[10] Matthias Eck, Stephan Vogel, Alex Waibel. Language Model Adaptation for Statistical
Machine Translation based on Information Retrieval, 2004, LREC
[11] Bing Zhao, Matthias Eck, Stephan Vogel. Language Model Adaptation for Statistical
Machine Translation with Structured Query Models, 2004, COLING[12] Masao Utiyama and Hitoshi Isahara. A Japanese-English patent parallel corpus. MTSummit XI, 2007[13] Masao Utiyama, Mikio Yamamoto, Atsushi Fujii, and Takehito Utsuro. Description ofPatent Parallel Corpus for NTCIR-7 Patent Translation Task. http://iflab.slis.tsukuba.ac.jp/fujii/ntc7patmt/ppc.pdf[14] Atsushi Fujii, Masao Utiyama, Mikio Yamamoto, Takehito Utsuro. Overview of thePatent Translation Task at the NTCIR-7 Workshop. Proceedings of the 7th NTCIRWorkshop Meeting on Evaluation of Information Access Technologies: Information
Retrieval, Question Answering and Cross-lingual Information Access, 2008.