The SAS Statistical Machine Translation System for WAT 2014

Rui Wang, Xu Yang and Yan Gao
SAS Institute IncBeijing, China
rui.wang|xu.yang|yan.gao@sas.com

nese and English into SVO languages. Our submission mainly focuses on using the syntactic approaches to improve effectively improve thetranslation results. In addition, unlike the alphabetic languages, Chinese and Japanese need to besegmented into words of characters before translation. The accuracy of segmentation highly influences the performance of translation for Chineseand Japanese. We apply the tokenization tool inSAS® Text Miner to the corpus and obtain improvement of the translation results.

Abstract
This paper is a description of the techniques and experiment results by SAS Institute Inc in WAT 2014 evaluation campaign. We participate in two subtasks ofWAT 2014: the Chinese to Japanese track
and the English to Japanese track. Our
baseline system is MOSES statistical machine translation toolkit. We propose syntactic reordering approaches for English
to Japanese and Chinese to Japanese translation which transform the order of source
sentence into the target-like order. In addition, we apply the segmentation tool inSAS® Text Miner to enhance the translation results. Several contrastive experiments are presented based on the automatic evaluation results.

1Japanese： 私は 日本語 を好きです 。
( I ) (Japanese) (like) (.)
SOVEnglish：English： IISS

likelikeVVJapaneseJapanese ..
OO

Chinese：喜欢(like)V日语 。
(Japanese) (.)
O我(I)SFigure 1: Word order differences of Japanese,
Chinese and English.

Introduction

This paper describes the machine translation
system employed by SAS Institute Inc in the 1stWorkshop on Asian Translation. We participate intwo subtasks in this year’s WAT evaluation campaign:
1) Chinese to Japanese;
2) English to Japanese.
We apply MOSES toolkit as the baseline system.
The sentence structure of Japanese is different
with that of English and Chinese. Japanese is typically a Subject-Object-Verb (SOV) language
while Chinese and English are Subject-Verb-Object (SVO) languages, as illustrated in Figure 1.
The statistic machine translation between Japanese and the SVO language is particularly difficult because of the long distance difference ofword orders. We propose a simple syntactic reordering approach for Chinese to Japanese and English to Japanese SMT in order to transform Chi-The rest of this paper is organized as follows:in section 2 we describe the background; in section 3 we present the architecture of our system;
Section 4 is the detailed description of the syntactic reordering rules we propose; experiment results are shown in section 5 and conclusion ismade in section 6.

2BackgroundThe state of the art Statistical Machine Translation(SMT), phrase-based SMT (Koehn et al., 2003),
works well on the translation between shortphrases as well as on long sentences pairs withsimilar word orders. However the phrase-based
SMT has limited capacity for long distance reordering since it does not consider the syntactic information in the translation. A great deal of recent
research enhances the translation results by adding syntactic features, such syntactic-based SMT64

Proceedings of the 1st Workshop on Asian Translation (WAT2014), pages 64‒68,
Tokyo, Japan, 4th October 2014.

(Liu et al., 2006), or forest-based SMT (Mi et al.,2008). The language-independent methods parsethe input sentences and then train reorderingmodel from these parsed trees, such as Quirk et al.
(2005), Li et al. (2007). These approaches improve the translation considerably but they aretime consuming during decoding. Syntactic reordering approaches effectively improve the translation results by transforming the order of sourceside language into target-like order.
Previous studies have proposed the syntactic
reordering approach which is regarded as a preprocessing step on the source side language. Xia
et al. (2004) automatically extract reordering rules
on source side for English to French translation.
Collins et al. (2005) improve the German to English translation by combining morphological andsyntactical information into SMT system. Wanget al. (2007) extract the syntactic reordering rules
manually on Chinese to English translation.
Isozaki et al. (2010) propose a simple head finalized approach to reorder English into Japanese or-der based on Enju parser. We apply the head finalized method to our system and achieve improvement on the translation from English to Japanese. Dan et al. (2012) carry out a similar head
finalized approach for Chinese to Japanese translation using a Chinese Enju parser (Yu et al.,2011). We propose a simple reordering methods
for Chinese based on the accurate Berkeley Parser(Petrov et al., 2006).

3We use the open-source SMT toolkit MOSES as
the baseline to train translation model and to decode the test set for submission. SRILM (Stolckeet al. 2002) toolkit is applied to train the N-gram
statistical language model. All the corpus are tokenized and lowercased before training and decoding. All the submitted results are detokenized.
In addition to the basic phrase-based model, we
also try hierarchical model (Chiang et al., 2007)
using MOSES.
We use the segementation tool in SAS® Text
Miner to tokenize Japanese and Chinese corpus(IP (NP (NR 本文 (this paper) ))(VP (VV 汇总 (summarize) )(AS 了)(NP (NP (CP (IP (VP (NP (NT 目前 (current) ))(VP (VRD (VV 查) (VV 到) (found) ))))
(DEC 的 (of) ))(NP (NN 影响 (influence) )))(CC 以及 (and) )(NP (CP (IP (VP (NP (NT 今后 (future) ))(VP (VV 预测 (predicted) ))))
(DEC 的 (of) ))(NP (NN 影响 (influence) )))))(PU 。))
本文 汇总 了目前 查到 的 影响 以及今后 预测 的System Architecture(IP(IP (NP(NP (NR(NR 本文
本文 (this(this paper)paper) ))))(VP
(VP (NP(NP (NP(NP (CP(CP (IP(IP (VP(VP (NP(NP (NT(NT 目前
目前 (current)(current) ))))
(VP(VP (VRD(VRD (VV(VV 查)查) (VV(VV 到)到) (found)(found) ))))))))(DEC(DEC 的的 (of)(of) ))))
(NP(NP (NN(NN 影响影响 (influence)(influence) ))))))(CC(CC 以及以及 (and)(and) ))(NP(NP (CP(CP (IP(IP (VP(VP (NP(NP (NT(NT 今后今后 (future)(future) ))))
(VP(VP (VV(VV 预测预测 (predicted)(predicted) ))))))))(DEC(DEC 的的 (of)(of) ))))
(NP(NP (NN(NN 影响
影响 (influence)(influence) ))))))))))(VV(summarize) ))(VV 汇总
汇总 (summarize)(AS(AS 了)了)(PU(PU 。))。))

影响 。

本文
本文 目前目前 查到
查到 的的

時点 で 検出されて いる 影響 および 今後 予測さ れる 影響 を まとめた 。

影响影响 以及以及 今后今后 预测预测 的的 影响影响汇总了汇总了 。
。

時点
時点 でで 検出されて検出されて いるいる 影響
影響 およびおよび 今後今後 予測さ予測さ れるれる 影響
影響 をを まとめたまとめた 。
。
(a)(IP (VP (PP (P 根据 (based on) )(NP (ADJP (JJ 大型 (large) ))(NP (NN 零售 (retail) ) (NN 商店 (shop) )(NN 选址法 (locating method) ))))(VP (VV 进行 (process) )(NP (NN 方针 (policy) ))(DEC 的))(NP (NN 修改 (modify) ))))(IP(IP (VP(VP (PP(PP (NP(NP (ADJP(ADJP (JJ
(JJ 大型大型 (large)(large) ))))
(NP(NP (NN(NN 零售
零售 (retail)(retail) )) (NN(NN 商店商店 (shop)(shop) ))(NN(NN 选址法选址法 (locating(locating method)method) ))))))))(P(based on)on) ))(P 根据
根据 (based(VP(VP (NP(NP (NN(NN 方针方针 (policy)(policy) ))))
(DEC(DEC 的))的))(NP(NP (NN(NN 修改修改 (modify)(modify) ))))))))(VV(process) ))(VV 进行进行 (process)

根据 大型 零售商店 选址法 进行 方针 的 修改大型大型 零售商店零售商店 选址法选址法 根据根据 方针
方针 的的 修改修改 进行进行

大規模 小売店舗 立地法 に 基づく 指針 の 改定 に ついて

大規模
大規模 小売店舗
小売店舗 立地法
立地法 にに 基づく基づく 指針
指針 のの 改定
改定 にに ついてついて(b)65for training, tuning and testing. Then we use MOSES tool to lowercase the corpus and to clean longsentences. The syntactic reordering approach isapplied on source-side cleaned sentences for English and Chinese. Finally the cleaned and reordered corpus is sent to MOSES to train models
and decode results.
We use Berkeley parser (Petrov et al., 2006) toparse the source side Chinese sentence into syntactic tree and develop the reorder rules based on
the grammatical structure of the sentence. English
sentence is parsed by Enju parser and the head finalized reorder is applied as described by Isozakiet al. (2010).

4

Syntactic Reordering Approach

4.1

Chinese to Japanese reordering

The Preposition (P) is always at the end thePrepositional Phrase (PP) in Japanese while in
Chinese P is located at the beginning of PP. We
imply the PP-rule, ‘PP (P (XXX)) → PP ((XXX)P)’, to move P to the end of PP. In Figure 2 (b),the preposition ‘根据 (based on)’ is move to theend of PP. The reordered sentence reduce the long
distance order differences between the source
Chinese and the target Japanese.
4.2

For English to Japanese translation, we use thehead finalization approach proposed by Isozaki etal. (2010). In Japanese, the syntactic head word islocated behind the dependent words while English
is the opposite. The dependency parser, Enju parser, is applied on the source side which can output
syntactic head, as illustrated in Figure 3. A simple
reorder rule is defined: to move the syntactic headto the end of non-head words. In Figure 3, nodeC3 is the head of node C2. According to the head
finalized reordering rule, C3 should be moved behind the other child of C2, C4. After reordering,
the English sentence has the similar order with theJapanese sentence.
The head finalized reorder rule will reorder thecoordination expressions such as, ‘A and B’, into
‘B and A’. We add a coordination exception rule
to stop reordering at coordination nodes.

The Chinese sentence is parsed into a grammaticaltree using Pen Treebank syntactic tagset as illustrated in Figure 2. The main difference between
Chinese and Japanese is the position of Verb
Phrase (VP) and Prepositional Phrase (PP). We
develop two simple reordering rules, VP-rule andPP-rule, based on the syntactic tree.
We develop the VP-rule, ‘VP (VV AS (XXX))→VP ( (XXX) VV AS)’, which means to movethe verb (VV) and the auxiliary word (AS) behind
VV to the end of the verb phrase (VP). For example, in Figure 2 (a), ‘VV 汇总 (summarize)’ and‘AS 了’ are moved to the end of VP. The reordered Chinese sentence has more similar orderwith the Japanese translation than the original one.

5Experiments

The experiment is conducted on the training datafrom ASPEC, consists of approximately 3 million

*

C0C0*

C1
C1

English to Japanese reordering

C0C0C2C2*

C1C1C3C4C4C2C2*C4C4C5C5C6C6**C5C5C6C6

C3

T1
T1

T2

T3
T3

T4
T4

T1
T1

T3
T3

T4
T4

T2

I

love

thechildrenI

thechildrenlove

私は子供 を愛していま

私は子供 を愛していますす(b) Reordered sentence(a) Original sentence

Figure 3: Example of reordering rules for English to Japanese translation. * represents the head ofthe node. (a) is the original sentence; (b) is the reordered sentence by head finalized rule.
66

Japanese-English parallel sentences and approximately 0.7 million Japanese-Chinese. BLEU andRIBES are used as evaluation metrics. On theevaluation websites, for Chinese to Japanese andEnglish to Japanese evaluation, the results ofBLEU and RIBES are measured under three Japanese segmentation tools: Juman, Kytea andMecab. For the sake of brevity, we calculate theaverage score of the three segmentation tools foreach evaluation metric.
5.1

VP rule and PP rule have contribution to the improvement of BLEU scores. We also apply thereorder rules to the hierarchical model and gain
2.07 in BLEU scores compared with the baseline
system.
5.3

The effect of the head finalized reordering rulefor English to Japanese translation is reported inTable 3. The phrase-based model is applied asbaseline system. After head finalized reordering,the BLEU score is increased from 28.52 by the organizer (Toshiaki, et al. 2014) to 31.09. We gain
3.13 in BLEU score by applying the hierarchicalmodel with pre-reordering.

Effect of the segmentation tool of SAS®
Text Miner

We use phrase-based model as baseline for theChinese to Japanese track to evaluate the effect ofSAS segmentation tool. The result of baseline isprovided by the WAT organizer (Toshiaki, et al.
2014). Table 1 shows that both the BLEU andRIBES are increased by using the segmentation
tool of SAS® Text Miner for Chinese and Japanese corpus compared with the Juman segmentation tool for Japanese and Stanford Word Segmenter for Chinese. We conduct the other experiments using the segmentation tool of SAS® Text
Miner because of its good performance.

Baseline
Baseline+reorder
Hierarchical
Hierarchical+reorder

Baseline
SAS segmentation

RIBES
0.769962
0.809631

Chinese to Japanese translation

Baseline
Baseline+VP
Baseline+PP
Baseline+PP+VP
Hierarchical
Hierarchical+PP+VP

BLEU
34.86
36.19
36.30
36.40
36.06
37.38

RIBES
0.690350
0.765005
0.743135
0.767323

ConclusionThis paper describes the techniques and experiment results in WAT 2014 evaluation campaign
submitted by SAS Institute Inc. By applying thetokenization tool in SAS® Text Miner and thesyntactic reordering rules, we achieve significant
improvements on Chinese to Japanese translationand English to Japanese translation. We also report some extensive experiment results to illustrate the contribution on different parts.
In the future, we intend to consider case markers in Japanese which other languages do not
have. Since the form of Japanese case markers isknown, we may define rules to insert particulartags to the source languages. We expect improvements of translation by adding the rules of casemarkers in Japanese.

Table 1: Effect of the segmentation tool of SAS®
Text Miner.
5.2

BLEU
28.52
31.09
31.23
31.65

Table 3: Effect of the head finalized reorderingrule for English to Japanese translation.

6BLEU
34.86
35.31

English to Japanese translation

RIBES
0.769962
0.826146
0.815694
0.826015
0.814207
0.830909

Table 2: Effect of reordering rules for Chinese toJapanese translation.

Reference

Table 2 shows the effect of reordering rules for
Chinese to Japanese translation. The baseline isthe results from the WAT organizer (Toshiaki, etal. 2014). We propose two reordering rules: VPrule and PP rule. The baseline is the phrase-basedmachine translation using MOSES. We apply theVP rule, PP rule and the combination of the tworules to the corpus separately and send the reordered sentences to the baseline system. Both theKoehn, Philipp, Franz Josef Och, and Daniel Marcu.
2003. "Statistical phrase-based translation." Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational
Linguistics on Human Language Technology-Volume 1. Association for Computational Linguistics.
Liu, Yang, Qun Liu, and Shouxun Lin. 2006. "Tree-tostring alignment template for statistical machinetranslation." Proceedings of the 21st International
Conference on Computational Linguistics and the67

44th annual meeting of the Association for Computational Linguistics.

Toshiaki Nakazawa, et al. 2014. "Overview of the 1st
Workshop on Asian Translation". Proceedings ofthe 1st Workshop on Asian Translation (WAT2014)

Mi, Haitao, Liang Huang, and Qun Liu. 2008. "ForestBased Translation." ACL.
Quirk, Chris, Arul Menezes, and Colin Cherry. 2005.
"Dependency treelet translation: Syntactically informed phrasal SMT." Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics.
Li, Chi-Ho, et al. 2007. "A probabilistic approach tosyntax-based reordering for statistical machinetranslation." ANNUAL MEETING-ASSOCIATIONFOR COMPUTATIONAL LINGUISTICS. Vol. 45.
No. 1.
Xia, Fei, and Michael McCord. 2004. "Improving a statistical MT system with automatically learned rewrite patterns." Proceedings of the 20th international conference on Computational Linguistics.
Association for Computational Linguistics.
Collins, Michael, Philipp Koehn, and Ivona Kučerová.
2005. "Clause restructuring for statistical machinetranslation." Proceedings of the 43rd annual meeting on association for computational linguistics.
Association for Computational Linguistics.
Wang, Chao, Michael Collins, and Philipp Koehn.
2007. "Chinese Syntactic Reordering for Statistical
Machine Translation." EMNLP-CoNLL.
Isozaki, Hideki, et al. 2010. "Head finalization: A simple reordering rule for sov languages." Proceedingsof the Joint Fifth Workshop on Statistical MachineTranslation and MetricsMATR. Association for
Computational Linguistics.
Dan, Han, et al. 2012. "Head finalization reordering for
Chinese-to-Japanese machine translation." Proceedings of the Sixth Workshop on Syntax, Semantics and Structure in Statistical Translation. Association for Computational Linguistics.
Yu, Kun, et al. 2011. "Analysis of the difficulties inChinese deep parsing."Proceedings of the 12th International Conference on Parsing Technologies.
Association for Computational Linguistics.
Stolcke, Andreas. 2002. "SRILM-an extensible language modeling toolkit."INTERSPEECH.
Chiang, David. 2007. "Hierarchical phrase-basedtranslation." computational linguistics 33.2: 201228.
Petrov, Slav, et al. 2006. "Learning accurate, compact,and interpretable tree annotation." Proceedings ofthe 21st International Conference on Computational Linguistics and the 44th annual meeting of theAssociation for Computational Linguistics. Association for Computational Linguistics.

68