English-to-Myanmar Statistical Machine Translation Using a Language
Model on Part-of-Speech in Decoding‡

Hnin Thu Zar Aye , †Chenchen Ding, ‡Win Pa Pa, ‡Khin Thandar Nwet†
Masao Utiyama, †Eiichiro Sumita†
Advanced Translation Technology Laboratory, ASTREC, NICT, Kyoto, Japan‡
Natural Language Processing Lab, University of Computer Studies, Yangon, Myanmar‡{hninthuzaraye,winpapa, khinthandarnwet}@ucsy.edu.mm,†{chenchen.ding, mutiyama, eiichiro.sumita}@nict.go.jp

most of the long-range reorderi-ngs found in thereference corpora for very diff-errrent word orderlanguages such as English and Myanmar. To recoverthat case, we used the paral-lel corpus which contains
short conversation sent-ences and will be described in
Section 5. Our motivation is to study that how POS
inf-ormation can support to SMT and affect the translation performance of SMT if we add POS to SMT.
We used PB SMT in this study because it is widelyused and offers as a phrase translation captures word
context and local reordering inher-ently in SMT [22].
We tuned the system with MERT algorithm [3] andused BLEU [16] and RIBES [17] to evaluate theperformance score. This study showed that PB SMT
with POS which used POS tagged language model(LM) could pr-oduce the improvement than thebaseline, the way to improve translation quality by
POS information. This paper is organized as follows:
Sect-ion 2 presents the nature of Myanmar language
and word segmentation for SMT. Section 3 expl-ains
the related works. Section 4 explains the POS tag set
and POS tagger and integrating POS to SMT. Section
5 presents about the experiment with corpus set up,
methodology and POS taggi-ng scheme on thetranslation. Section 6 presents the conclusion andfuture work.

Abstract
In this study, we investigate English-to-Myanmar statistical machine translation (SMT) by using
part-of-speech (POS) as linguistic inform-ation
which is valuable for statistical machine translation
systems to extract translation rules. This paper
presents the POS tagger which is also used as
segmenter, how to add POS to the phrase-based SMT
and the experiment. The experimental results showedthat the baseline SMT with POS that used POS
tagged language model could outperform thebaseline system in terms of BLEU and RIBES scores.

1. Introduction
The state-of-the-art machine translation (MT)methods apply statistical techniques to extract translation rules automatically from parallel data.
Phrase-based (PB) statistical machine translation(SMT) (PB SMT) model and decoder have been
introduced and showed that phrase translation cangive better performance than word-based transl-ation
models [1]. Then, the operations of transl-ation
models bases on not only words but also moregeneral linguistic representations such as le-mma,
part-of-speech (POS). The available of that
information to the translation model allows the direct modeling of many aspects of translation onmorphology, syntax or semantics. The phrase-basedtranslation models with additional linguistic
information are called factored translation models[2].
Although these SMT systems can be applied
for similar languages pairs, it is still a challenging
task in automatic MT for different word orders
language pairs. English is Subject-Verb-Object(SVO) and Myanmar is Subject-Object-Verb (SOV).
The reordering model of the PB SMT is suitable for
local reordering. However, it cannot solve to capture

2. Myanmar Language
This section will briefly explain about thenature of word segmentation and word order inmachine translation for Myanmar language.
Myanmar words composed of single or multiple
syllables are usually not separated by white space. In
addition, there is no consistent rule for word
segmentation even though spaces are used in
Myanmar sentences to segment between phrases andwords which can give clear and correct meaning.

409

It has been clearly described for sentencecomposing in Myanmar grammar. But there are manyrules for sentence structure and they can be used
more freely than English. Moreover, some of the
English POS like determiner, auxiliary verb, some
prepositions are not used in Myanmar and some
Myanmar POS like particles and post positi-onal
markers are not used in English. So, word reordering
is one of the problems for statistical machinetranslation between Myanmar and English as we havedescribed in Introduction. Example of English to
Myanmar translation will be illustrated in Figure 1
with the order of words of each sentence and distinct
POS of words in segmented sentences.

The hierarchical phrase-based statistical machine translation (HIERO) and the operation sequence model (OSM) SMT have been carried out for
automatic translation on BTEC corpus from Myanmar to English and other twenty languages by using
three methods of Myanmar word segment-ation as we
described in Section 2 [8]. These SMT methods havebeen carried out on BTEC corpus between English
and the under-resourced langu-ages which are
Laotian, Myanmar and Thai [10].
Empirical dependency-based head finalizat-ion
for statistical machine translation between Chinese,
English, French, and Myanmar have been presented
by using dependency parser based on source
languages dependent features with the default the
distortion limit (DL) value and alterna-ting the DL
values in decoding step to analyze the SMT
reordering [5].
Factored machine translation for Myanmar to
English, Japanese and vice versa have been presented. They did preprocessing to add POS tag tosource and target path and language model build-ing
for PBSMT baseline to be factored translation
models. They explained how to insert the POS
information between source and target language withtranslation factors. They compared the trans-lation
performance results based on translation factors bymeans of BLEU terms. They also des-cribed thatmappings involving POS tags between the languages
are more effective in Myanmar to Japan than English
to Myanmar and Japanese to Myanmar. Furthermore,they also presented that English to Myanmar andJapanese to Myanmar translations were difficult toget better score on baselines because of the quality of
POS tagger and using different domain tagger.
Moreover, they reported that the factored SMT could
give higher BLEU scores for Myanmar to Japan and
Myan-mar to English translation, the computational
exp-ense was especially high when two translation
pa-ths approaches were used. Finally, they suggested
that although the factored machine translation for the
low-resource language Myanmar could give higher
translation performance than typical PB SMT, it
needs to choose carefully the type of factored model
used [11].
The above related work [10] has shown that
PB SMT provided the better BLEU score than other
SMT systems. Then the related work [11] haspresented the POS tag information is useful fortranslation between Myanmar to English and Japan.
Therefore, we decided to use PB SMT and
POS tag information to be added to PB SMT to

Figure 1.Example of word segmentation
and word order between Myanmar and English
Myanmar word segmentation methods havebeen proposed based on syllables, maximum matching the longest string, statistical approaches andmachine learning approaches in [6] and [9]. The
sentences of the corpus should have the corr-ect
sentence structure to be supported the suffice-ent
facility in word segmentation and the correct POS
information to SMT. Furthermore, it is imp-ortant tofollow correct grammar order with con-sistent format
in composing of Myanmar senten-ces if they will be
used in SMT.

3. Related Work
There have been studies on English-to-Myanmar automatic statistical machine translation. We will
describe some of these studies.
The string-to-tree (S2T) and tree-to-string(T2S) statistical machine translations for Myan-marto-Chinese, English, French, and German vice versa
tested on BTEC corpus [18]. They evaluated with
automatic evaluation metric and human evaluation
[7].

410

CRF++ toolkit1 for training POS tagger and word
segmentation in our experiment.

investigate the performance of translation between
English to Myanmar by using the POS tagger.

Table 1.

4. Methodology
4.1. NOVA POS for Myanmar Language

NOVA POS tag set

Tag

We used NOVA POS tag set because it could
provide flexible annotation for tokenization. The POS
tag set defines seven POS tags which are four basic
main POS tags and three POS tags to be used in
Asian Language Treebank (ALT) project. The
detailed description will be described in Table 1. The
NOVA POS tag set can provide the POS tag
annotation of compound words and suffixes indetailed by using brackets (“[” and “]”) called patterntag.
As Myanmar language is analytic, most ofnouns, adjectives and verbs are usually suffixed oraffixed with post positional markers (ppm) or
particles. In addition, it has many compound words.
For that cases, it is enable to tag each word easily andannotate by brackets as pattern tag by placing the
POS tag of compound word before and end of thepattern as T [t1 t2 … t3] T where T is POS ofcompound or transformed word which might be nounor adjective or verb and sequence tag list: “ t1 t2… t3” in bracket is POS of each word in compound ortransformed word. Particles and post positional
markers (ppm) are usually suffixed to Myanmar
verbs which can be tagged by pattern tag and theexample verb pattern tag will be illustrated in Figure
2.
Numeral amount of nouns are tagged by numeral patterns which are assumed as adjective of nouns
and tagged as “a”, a[ t1 t2 ]a , and “t1” is tagged as“1” for numeral letter or number and “ t2” is tagged
as “n” , by assuming as noun for type classifier
particles respectively. Example tagging of numeralamount will be shown in Figure 3.

n
v
o
a
1

Description
noun
verb
other which might be adverb, post
positional markers(ppm), particles,
conjunctions
adjective
numeral letter and numeric number.

punctuation marks+

for all tokens with weak syntactic roles
such as interjection

Figure 1. Verb pattern annotation

Figure 2. Numeric pattern annotation

4.2. Tagged by CRF

We trained POS tagger with CRF++ toolkit byusing the feature set of token unigram at their relative
position -1 , -2 , 0 , +1 , and +2 as {w-2 , w-1 , w0 , w+1, w+2 } where w is defined as the to-ken of eachposition and the related label, y, of each token is { y-2,y-1 , y0 , y+1 , y+2 }. These n-grams tokens are
combined with label unigram to produce the feature
set for the model.
For example sentence, “General POS tagging methods for Myanmar
have been proposed for Myanmar language. Myanmar language has been still in under resource
language and there is no standard word segmen-ter
and POS tagger.
The conditional random field (CRF) [20]machine learning approach is the widely used forsegmenting/labeling sequential data. General machine learning framework has been reported sufficiently powerful to handle the Myanmar word
segmentation task [9]. Thus, we used statistical”, will be segmented into four tokens:
1411

http://taku910.github.io/crfpp/

,

,

,

the data sparseness problem caused by limited
training data [1].
Thus, for our study, we did as we mentioned
in Section 4.2 to get BTEC1 POS tagged train data as
shown in figure 5. Then we used the translational
factor to integrate POS into PB SMT to training
target language side.

and POS tagged to each word line by linein the trained data preparation for the train format of
CRF. Word and POS tagged paired lines as rows andcolumns can be viewed as a table for the unigramfeature set tokens of the trained sentence as
illustrated in Figure 4 where i repre-sents their row
positions of token, w and POS label, y.
For the unigram feature, the feature function, f(w-1 , y0) , will return 1, if w-1 is “

” and theunigram label output, y0 , is “ v[v ”. After training
the POS tagger with CRF, the train data has been
segmented and tagged as the form of NOVA POS tag
set.

Figure 4. Example of lexical entry
Moreover, we added POS to LM by using POS
tagged corpus which is result from CRF++ POS
tagger. The POS tagged train data was used withuntagged and POS tagged lan-guage models in
decoding which will be discussed in next Section.

Figure 3. Example of unigram feature

5. Experiment
4.3 Integrating POS into PB SMT
5.1. Corpus and Setting
Phrase-based translation models allow lexical
entries with more than one word on either the source-language or target-language side: for exam-ple,we might have a lexical entry (, the tree)
specifying that the string,

In this experiment, we used Basic Travel Expression Corpus (BTEC1). The corpus was randomly divided into three data set for train, development (dev), and test and their statistic will be shownin Table 2. The train data set was used to train SMT
systems and the development (dev) data set was usedin tuning the system. Then the test data set was used
to evaluate the SMT system.
We used PB SMT system of Moses toolkit2[12] as a baseline system. GIZA++ [13] is used forword alignment between source and target lan-guagealignment was symmetrized by grow-diag-final-andheustristics [1]. The lexical reordering model wastrained with the msd-bidirectio-nal-fe option [14].
The maximum phrase length is 9. To train the
9-gram language models (LM) with interpolated
modified Kneser-Ney discounting [19], we used
SRILM [15].

” in Myanmar

can be translated as “the tree” in Eng-lish. The optionof having multi-word expressi-ons on either thesource or target-language side is a signiﬁcantdeparture from word-to-word transl-ation models.
Multi-word expressions are useful in translation for
the improvements of phrase-based translation
models. A phrase pair is a pair ( s , t ), ( s' , t' )where ( s , t ) is a subsequence within the source
language sentence, and ( s' , t' ) is a subsequence
within the target language sen-tence [25]. For
example, consider the trained case where k is thelength of sentence and source, f, and target, e,language consists of the following sentences asshown in Figure 5. Then, ( s , t ) = (1, 2) , ( s' , t' )= ( 3, 4) would correspond to the potential lexical
entry:( Where is,Table 2. Statistics on data set)

Additional linguistic information which wementioned in Introduction could be tightly inte-grated
to the phrase-based translation models as translation
factors. They are valuable for better explanation formany aspects of translation as the performance ofstatistical machine translation models can overcome
2412

Data set#Sentences#Tokens
EnglishMyanmarTraining
Development
Test

159,603

948,611625,086

1,622

9,748

6,413

973

5,791

3,836

http://www.stamt.org.moses

same. All systems could prod-uce correct outputs for
short sentences like the first sentence of Figure 6.
However, all systems could not well translate longsentences. Among these incorrect results, the phrase
order of second and third SMT systems is better thanthe baseline as the second and third samples of Figure
6. But the related phrase orders of third SMT’s
results are better than the second one’s.
Table 3. Scores of the SMT systems
SMT systems
Baseline + LM
Baseline + POS + LM
Baseline + POS+ POS LM

Figure 5. Translation examples
For decoder setting, we used default parameters setting of Moses decoder except using distortion-limit value 12 to be occurred sufficient supp-ort
from the LM. We used two language models:
untagged (LM) and POS tagged (POS LM) for threesystems’s experiments as shown in Table 3. The firstbaseline SMT used LM. The second baseline SMT
was added POS to target language training and used
LM. The third baseline SMT was added POS totarget language training and used POS LM.
We used minimum error rate tuning (MERT)
to tune the decoder parameters. We evaluated thetranslation results by two automatic measures:
bilingual evaluation understudy (BLEU) which isused to measure the adequacy of the translations andrank-based intuitive bilingual evaluation measures(RIBES) which will penalize the wrong word orders.
The higher BLEU score and the larger RIBES
indicate the better performance.

BLEU
.409
.409
.410

RIBES
.564
.564
.574

5.3. Discussion
The result of BLEU and RIBES of the third
SMT which used POS on LM and train data was alittle improvement on baseline and baseline with POS
train data that used LM. BTEC1 corpus has thecolloquial style short sentences for travel dom-ain.
ALT corpus has literature style long translat-ed
sentences for news domain. Word segmentati-on
forms of these two styles are different. ALT POS
tagger could tag and segment flexibly the literaturesentences. However, ALT POS tagged model could
lightly segment and add POS inform-ation to BTEC1
trained data. The reason might depend on the above
conditions. The third SMT could produce 468 outputs
as the reference while others produced 461 outputs
for 973 test sentence-s. The baseline without using
POS could not hand-le for long phrases and foreign
words translations. This condition is also in the third
SMT. However, among incorrect outputs of all
systems which are different from the references, themeaning of wor-ds and related phrase orders of thesentences of the third SMT system are better than thefirst and second ones although they are not totallycorrect.

5.2. Results
The experimental results of English to
Myanmar translation on baseline SMT and SMT systems with POS will be listed in Table 3. The scores ofthe first and second systems are the same in terms of
BLEU and RIBES although the second one used POS
tagged trained data. However, it can be clearly seen
the third SMT system which used POS LM in
decoding and POS tagged train data was higher than
the first and second ones in terms of BLEU and
RIBES although the high score point is a little. The
next Section 5.3 will be discu-ssed about why theevaluation scores of systems with POS could notincrease much. Three sample results from all SMT
systems with original transl-ated references areshown in Figure 6 by showing the results of second
and third systems from Table 3 which used POS
tagged trained data in decoding as the baseline plus
POS because their correct out-puts are nearly the

6. Conclusion and Future Work
We have presented the condition of Myanmar
language for SMT and how to integrate POS to the
PB SMT. And then we have also discussed about theeffect of using different domain corpus for POS
tagging and word segmentation and the resu-lt
statement of the experiment. We could find that using
POS information in SMT that can be able to improve
the translation performance even though the results of413

evaluation metrics of this study are not muchdifferent in three experiments.
For future work, we intend to test PB SMT
with POS information by doing more preprocess-ing
steps such as word segmentation to know that the
POS information how much support to SMT and how
can support to produce more better performancebased on the better translation result of this study.

tree to String Statistical Machine Translation
for Myanmar Langua-ge”, Proceedings of
ICCA2016, Yangon, Myanmar, February 2526, 2016, pp.249-257.
[8] Ye Kyaw Thu, A. Finch, Win Pa Pa, and E.
Sumita, “A Large-scale Study of Statistical
Machine Translat-ion Methods for Myanmar
Language”, In Proceedings of SNLP2016,
Phranakhon Si Ayutthaya, Thailand, February
10-12, 2016.
[9] C. Ding, Ye Kyaw Thu, M. Utiyama and E.
Sumita, “Word Segmentation for Burmese(Myanmar)”, ACM Transactions on Asian and
Low-Resource
Language
Information
Processing (TALLIP)Volume 15, No. 4, Article
22, May 2016, pp. 22:1-22:10.
[10] Win Pa Pa, Ye Kyaw Thu, A. Finch, and E
Sumita, “A Study of Statistical MachineTranslation Methods for Under Resourced
Languages”, 5th Workshop on Spoken
Language Technologies for Under-resourced
Languages (SLTU Workshop), Procedia
Computer Sc-ience, Volume 81, 2016,
Yogyakarta, Indonesia, May 09-12, 2016, pp.
250-257.
[11] Ye Kyaw Thu, A. Finch, Akihiro Tamura, E
Sumi-ta and Y. Sagisaka, “Factored MachineTranslation for Myanmar to English, Japaneseand Vice Versa”, Proc-eedings of the 12th
International Conference on Com-puter
Applications (ICCA 2014), Yangon, Myanmar,
February 17-18, 2014, pp. 171-177.
[12] P. Koehn, H. Hoang, A. Birch, C. CallisonBurch, M. Federico, N. Bertoldi, B. Cowan,
W. Shen, C. Mor-an, R. Zens, C. Dyer, O.
Bojar, “Moses: Open source toolkit for
statistical machine translation”, Proceeding of
ACL, 2007, pp. 177-180.
[13] F. J. Och and H. Ney, “A systematiccomparison of various statistical alignment
models.”, Computation-al Linguistics, vol. 29,
no. 1, 2003, pp. 19-51.
[14] C. Tillmann, “A unigram orientation model for
sta-tistical machine translation.”, Proceedingsof HLT-NAACL 2004, Association for
Computer Linguistic, Boston, Massachusetts,
USA, May 02-07, 2004, pp. 101- 104.
[15] A. Stolcke, “SRILM–an extensible language
mod-eling toolkit”, Proceedings International
Conference on Spoken Language Processing,November, 2002, pp. 257-286.

References[1] P. Koehn, F. J Och, and D. Marcu, “StatisticalPhrase-Based Translation”, Proceedings of
HLT-NAA-CL 2003, Edmonton, Canada, May
27-June 1, 2003, pp. 48-54.
[2] P. Koehn and H. Hoang, “Factored Translation
Models”, Proceedings of the 2007 Joint
Conference on Empirical Methods in Natural
Language Processing and Computational
Natural Language Learning, Prag-ue, Czech
Republic, June 28-30, 2007, pp. 868-876.
[3] F. J Och, “Minimum error rate training instatistical machine translation”, Proceedings ofthe 41st Annual Meeting on Association for
Computational Linguistics – Volume 1,
Association for Computer Linguistics,
Sapporo, Japan, July 07-12, 2003, pp. 160167.
[4] Ye Kyaw Thu, A. Finch, E. Sumita and Y.
Sagisaka, “Integrating Dictionaries into an
Unsupervised Model for Myanmar Word
Segmentation”, Proceedings of the 5thWorkshop on South and Southeast Asian NLP(WSSANLP 2014), Dublin, Ireland, August 2329, 2014, pp. 20-27.
[5] C. Ding, Ye Kyaw Thu, M. Utiyama, A.
Finch, and E. Sumita, “Empirical DependencyBased Head Final-ization for Statistical
Chinese-, English-, and French-to-Myanmar(Burmese) Machine Translation”, Proceedings of the 11th International Workshop on
Spoken Language Translation (IWSLT 2014),
Lake Tahoe, CA, USA, December 4-5, 2014,
pp.184-191.
[6] Win Pa Pa, Ye Kyaw Thu, A. Finch, and E.
Sumita, “Word Boundary Identification for
Myanmar Text Usi-ng Conditional Random
Fields”, Proceedings of the Ninth International
Conference on Genetic and Evolut-ionary
Computing, Yangon, Myanmar, August 26-28,
2015, pp. 447-456.
[7] Ye Kyaw Thu, A. Finch, Win Pa Pa, Khin War
War Htike and E. Sumita, “String to Tree and414

[16] K. Papineni, S. Roukos, T. Ward, and W. J.
Zhu, “BLEU: a method for automatic
evaluationofmachinetranslation.”,Proceedings of the 40th Annual Meeting on
Association for Computer Linguistics ACL’02,
Ass-ociationfor
Computer
Linguistic,
Philadelphia, Penn-sylvania, USA, July 07-12,
2002, pp. 311-318.
[17] H. Isozaki, T. Hirao, K. Duh, K. Sudoh, and H.
Tsukada, “Automatic evaluation of translation
quality for distant language pairs”, Proceedingsof Empirical Methods in Natural Language
Processing, EMNLP’10, Association for
Computer Linguistic, Cambridge, Massachusetts, USA, October 09-11, 2010, pp. 944952.
[18] G. Kikui, E. Sumita, T. Takezawa, and S.
Yama-moto, “Creating corpora for speech-tospeechtranslat-ion”,
Proceedings
ofEUROSPEECH 2003-INTERSP-EECH 2003,
Geneva, Switzerland, September 1-4, 2003, D,
pp. 381–384.
[19] S. F. Chen, and J. Goodman, “An empirical
study of smoothing techniques for language
modeling”, Proc-eedings of the 34th annual
meeting on Association for Computational
Linguistics, Association for Computati-onal
Linguistics, Santa Cruz, California, USA, June
24 -27, 1996.
[20] J. Lafferty, A. McCallum, and F. Pereira.,“Cond-itional random fields: Probabilistic
models for segm-enting and labeling sequencedata”, Proceedings of the EighteenthInternational Conference on Machine Learning
ICML’01,
MorganKaufmann,Williamstown, MA, USA, June 28-July 01,
2001, pp. 282-289.
[21] F. Sha and F. Pereira, “Shallow parsing withconditional random fields”, Proceedings of HLT/ NAACL’03, Edmonton, Canada, May 27-June
01, 2003, pp. 134-141.
[22] H. Setiawan, H. Li1, M. Zhang, and B. Chin
Ooi, “Phrase-Based Statistical MachineTranslation: A Lev-el of Detail Approach”,
Proceedings of the Second international joint
conference on Natural Language Processing(IJCNLP’05), Jeju Island, Korea, October 1113, 2005, pp. 576-587.
[23] Ye Kyaw Thu, Win Pa Pa, Masao Utiyama, A.
Finch and E. Sumita, “Introducing the Asian
Language Treebank (ALT)”, 10th edition of
the Language Resources and Evaluation
Conference, Portorož, Slo-venia, May 23-28,
2016.
[24] H. Riza, M. Purwoadi, Gunarso, T.
Uliniansyah, A.A. Ti, S. M. Aljunied,
L.C.Mai, V. T. Thang, N. P. Thai, R. Sun, V.
Chea, Khin Mar Soe, Khin Thandar Nwet, M.
Utiyama, C. Ding, “ Introduction of the Asian
Language Treebank”, Conference of The
Oriental Chapter of International Committee,
Bali, In-donesia, October 26-28, 2016.
[25] M. Collins, “Phrase-Based Translation
Models”, April 10, 2013415