Supervised and Unsupervised Machine Translation
for Myanmar-English and Khmer-English
Benjamin Marie
Hour Kaing
Aye Myat Mon
Chenchen Ding
Atsushi Fujita
Masao Utiyama
Eiichiro Sumita
National Institute of Information and Communications Technology
3-5 Hikaridai, Seika-cho, Soraku-gun, Kyoto, 619-0289, Japan{bmarie,hour_kaing,ayemyatmon,chenchen.ding}@nict.go.jp{atsushi.fujita,mutiyama,eiichiro.sumita}@nict.go.jp
Abstract

2002). We also show that the use of monolingual data can dramatically improve translation quality and that an advanced cleaningand normalization of the data further boosts
the translation quality. For contrastive experiments, and for investigating the feasibilityof unsupervised machine translation (MT) for
low-resource distant language pairs, we alsopresent unsupervised MT systems that only
use for training the development data provided
for these tasks and our monolingual data.
The remainder of this paper is organized asfollows. In Section 2, we introduce the data
preprocessing, including cleaning and normalization steps. In Section 3, we describe thedetails of our NMT and SMT systems. The
back-translation of monolingual data used bysome of our systems is described in Section 4.
Then, the combination of NMT and SMT isdescribed in Section 5. In Section 6, we present
our unsupervised MT system. Empirical results achieved by all our systems are showed
and analyzed in Section 7. Section 8 concludes
this paper.

This paper presents the NICT’s supervisedand unsupervised machine translation systems for the WAT2019 Myanmar-English
and Khmer-English translation tasks. For
all the translation directions, we built
state-of-the-art supervised neural (NMT)and statistical (SMT) machine translation
systems, using monolingual data cleaned
and normalized. Our combination of NMT
and SMT performed among the best systems for the four translation directions.
We also investigated the feasibility of unsupervised machine translation for lowresource and distant language pairs andconfirmed observations of previous work
showing that unsupervised MT is stilllargely unable to deal with them.

1

Introduction

This paper describes neural (NMT) and statistical machine translation systems (SMT) built
for the participation of the National Instituteof Information and Communications Technology (NICT) in the WAT2019 (Nakazawa
et al., 2019) Myanmar-English (my-en) and
Khmer-English (km-en) translation tasks.1
We present supervised systems built using theparallel data provided by the organizers andexternal additional monolingual data. For all
the translation directions, we trained supervised NMT and SMT systems, and combined
them through n-best list reranking using several informative features (Marie and Fujita,
2018a), as in our previous participation to
WAT2018 (Marie et al., 2018). This simple
combination method achieved the best results
among the submitted MT systems for thesetasks according to BLEU (Papineni et al.,1

2 Data preprocessing
To train our systems, we used all the bilingual
data provided by the organizers. The provided
bilingual data comprises different types of corpora: the training data provided by the ALT
project2 and additional training data. These
additional data are the UCSY corpus, constructed by the University of Computer Studies, Yangon (UCSY),3 for the my-en task, andthe ECCC corpus, collected by National Institute of Posts, Telecoms & ICT (NIPTICT)
2
http://www2.nict.go.jp/astrec-att/member/mutiyama/ALT/index.html
3Note that this corpus is not the same as last year
and has been further cleaned by the organizers.

The team ID of our participation is “NICT-4”.

68
Proceedings of the 6th Workshop on Asian Translation, pages 68–75
Hong Kong, China, November 4, 2019. c 2019 Association for Computational Linguistics

and cleaned by NICT, for the km-en task.
For English, we used the monolingual corpora provided by the WMT18 shared News
Translation Task (Bojar et al., 2018). For
Khmer, we experimented with a monolingual corpus extracted from Common Crawl.4
As for Myanmar, we experimented with twomonolingual corpora: Myanmar Wikipedia
and Myanmar Common Crawl. During ourlast year’s participation in the task, we only
observed slight improvements, or even a significant drop of the translation quality with
the Common Crawl corpus, when using these
Myanmar monolingual corpora that we assumed to be the consequence of the extreme
noisiness of the data. This year, we introduce a new cleaning and normalization process(Section 2.1) to better exploit the monolingual data. The Wikipedia corpus was createdfrom the entire Myanmar Wikipedia dumped
on 2017/06/01. The Khmer and Myanmar
Common Crawl corpora consist of sentences intheir respective languages5 from the first quarter of the Common Crawl data crawled during April 2018 for Myanmar, and April 2019
for Khmer. These monolingual corpora, especially the Common Crawl corpora crawledfrom various websites, contain a large portionof useless data that necessitates cleaning andnormalization as presented in the Sections 2.1
and 2.2.
We tokenized and truecased English data respectively with the tokenizer and truecaser ofMoses6 (Koehn et al., 2007). The truecaser
was trained on all our English monolingual
data. Truecasing was performed on all the tokenized data. For Myanmar, the provided bilingual data were already tokenized. However,for the sake of consistency with our tokenizer
we chose to reverse it and tokenized the bilingual and monolingual data by ourselves withan in-house tokenizer. We did not apply truecasing to the Myanmar data. We performed
the same procedure for Khmer.
For cleaning, after pre-processing the Myanmar and Khmer monolingual data as describedin the Sections 2.1 and 2.2, we segmented thetext into sentences and removed lines in both
corpora that fulfill at least one of the following
conditions:• more than 25% of its tokens are numbers
or punctuation marks.
• contains less than 4 tokens• contains more than 80 tokens
For cleaning bilingual data, we only applied
the Moses script clean-n-corpus.perl to remove lines in the parallel data containing morethan 80 tokens and escaped characters forbidden by Moses. Note that we did not perform
any punctuation normalization.
To tune/validate and evaluate our systems,
we used the official development and test sets
designated for the tasks: the ALT test dataconsisting of translations of English texts sampled from English Wikinews.
Tables 1 and 2 present the statistics of theparallel and monolingual data, respectively,
after preprocessing.
2.1 Cleaning of Myanmar Data
Many lines in the Common Crawl corpus are
made of long sequences of numbers and/or
punctuation marks, and 80% of Myanmar lines
are not written in a standard Unicode format. It also contains foreign languages, suchas English, Thai, and Chinese sentences. In
the Wikipedia corpus, a standard Unicode format is used but the text is also very noisy.
The most common issues in these corpora arespelling errors. From these observations, weapplied the following steps for cleanings:• Encoding normalization• Noisy sentence removal• Spelling error correction
First, we used the UCSY encoding converterto convert Zawgyi font to Unicode.7 Second,
we manually removed 22% of noisy sentencesin the Common Crawl corpus and 15% of noisy
sentences in the Wikipedia corpus.
There are many spelling errors in the corpora. The spell and pronunciation of a word4

https://commoncrawl.org/
We used fasttext and its pretrained models forlanguage identification: https://fasttext.cc/blog/
2017/10/02/blog-post.html
6https://github.com/moses-smt/mosesdecoder
5

7This step requires three minutes of computationaltime for processing one thousand sentences.

69

#sent. pairs (#tokens)

Data set
Train
Development
Test

my-en

km-en

221.1k (my: 4.1M, en: 3.2M)
1,000 (my: 36,688, en: 25,538)
1,018 (my: 37,519, en: 26,236)

122.7k (km: 4.1M, en: 3.3M)
1,000 (km: 33,604, en: 25,538)
1,018 (km: 34,238, en: 26,236)

Table 1: Statistics of our preprocessed parallel data.

CorpusWMT (English)
CommonCrawl (English)
Wikipedia (Myanmar)
CommonCrawl (Myanmar)
CommonCrawl (Khmer)#lines338.7M
2.0M
268.7k
3.0M
882.9kOrder
12#tokens
7.5B
44.5M
5.5M
67.5M
30.1M

To

Graph

◌្ + ដ

◌្ + ត

◌�◌ំ + ◌ា◌ា + ◌ំ◌ាំ◌៉ + ◌ាំ◌ុាំS[S] + V
SS + WS-◌ុ + ◌ាំ3◌ាំ + ◌ុ

4
5

V + S[S]WS + SS◌ាំ + ◌៉

Table 2: Statistics of our preprocessed monolingual data.

Table 3: Khmer Text normalization rules, where”V” is Vowel, ”S” is subscript (subscript sign + a
consonant) and [S] refer to one or zero subscript,
WS is west subscript, and SS is south subscript.

may lead to misspelling because there are complex orthographic rules and a large gap between the script and the pronunciation in the
Myanmar language. One type of spelling errors results in words that do not exist in the
Myanmar language. They can be detected easily by a spell checker and a dictionary lookup. Another type of errors happened when
the writer uses existing words but wrongly or
ambiguously in context. Those errors are difficult to automatically detect as these words
exist in a Myanmar dictionary but are incorrect according to the context. There are two
types of errors: phonetic errors and context
errors. Context error is a subset of phonetic
error (e.g., “I saw three trees in the park” as“I saw tree trees in the park”).
We performed a dictionary8 look-up to
match the word in the given text with the wordin the dictionary. If a word is not there thenit is considered as an error. We also measured
the Levenshtein distance at the character levelto find the closest word in a large Myanmar
dictionary. After generating a list of suggestions, we used a bigram language model to select and apply the best correction in context.
2.2

From

characters that should never be separated bya space.
The Khmer corpus is in Unicode format
and it is very common for spelling ambiguities
where multiple character sequences can represent word with the same graphical representation. We solve this problem by replacing thespelling ambiguities into one form which basically follows the way of Khmer native speakers’ spelling. The replacement rules are simply
in the order as in Table 3.
As our in-house tokenizer works at character level, over-tokenization is unavoidable
when out-of-vocabularies (OOVs) appear. We
reverted the over-tokenization by removing
spaces as follows:• before [U+17B6 - U+17D3]• before .?[U+17CB - U+17CD]• before and after U+17D2

Cleaning of Khmer Data

• after [U+17A5 U+17A7 U+17AB U+17AD]

We clean the Common Crawl corpus for
Khmer in two steps, spelling disambiguationand over-tokenization recovery. In our context, over-tokenization refers to dependant

However, recovering from over-tokenization
did not result in improvements of translation
quality according to BLEU. Consequently, forthe sake of simplicity, we did not use this stepwhen building our MT systems.

8

We used a Myanmar dictionary that contains alist of unique 41,343 Myanmar words from https://github.com/chanmratekoko/Awesome-Myanmar.

70

right) lexicalized reordering models. We also
used the default distortion limit of 6. We
trained two 4-gram language models, one on
the WMT monolingual data for English, on
the Common Crawl corpus for Khmer, andon the Wikipedia data for Myanmar, concatenated to the target side of the parallel data,
and another one on the target side of the parallel data, using LMPLZ (Heafield et al., 2013).
To tune the SMT model weights, we used
kb-mira (Cherry and Foster, 2012) and selected the weights giving the best BLEU score
for the development data during 15 iterations.

--type transformer --max-length 80--mini-batch-fit --valid-freq 5000--save-freq 5000 --workspace 10000--disp-freq 500 --beam-size 12--normalize 1 --valid-mini-batch
16 --overwrite --early-stopping
5 --cost-type ce-mean-words--valid-metrics ce-mean-words
perplexity translation --keep-best--enc-depth 4 --dec-depth 4--transformer-dropout 0.1--learn-rate 0.001 --dropout-src
0.1 --dropout-trg 0.1 --lr-warmup
16000 --lr-decay-inv-sqrt 16000--lr-report --label-smoothing
0.1 --devices 0 1 2 3 4 5
6 7 --dim-vocabs 8000 8000--optimizer-params 0.9 0.98
1e-09 --clip-norm 5 --sync-sgd--exponential-smoothing

4 Back-Translation of Monolingual
Data for NMT
Parallel data for training NMT can be augmented with synthetic parallel data, generated through a so-called back-translation, tosignificantly improve translation quality (Sennrich et al., 2016a). We used an NMT system,
trained on the parallel data provided by the organizers, to translate target monolingual sentences into the source language. Then, theseback-translated sentences were simply mixed
with the original parallel data to train fromscratch a new source-to-target NMT system.
We back-translated 2M sentences randomly
sampled from WMT18 English data for
my→en and km→en, our Myanmar Wikipedia
corpus for en→my, and our Khmer Common
Crawl corpus for en→km.

Table 4: Parameters of Marian used for training
our NMT systems.

3

Supervised MT Systems

3.1

NMT

To build competitive NMT systems, we relied on the Transformer architecture (Vaswani
et al., 2017). We chose Marian9 (JunczysDowmunt et al., 2018) to train and evaluate
our NMT systems. In order to limit the sizeof the vocabulary of the NMT models, we further segmented tokens in the parallel data into
sub-word units via byte pair encoding (BPE)(Sennrich et al., 2016b) using 8k operations
for each language. All our NMT systems wereconsistently trained on 8 GPUs,10 with the parameters presented in Table 4.
3.2

5 Combination of NMT and SMT
Our primary submissions for the tasks were
the results of a simple combination of NMT
and SMT. As demonstrated by Marie andFujita (2018a), and despite the simplicity ofthe method used, combining NMT and SMT
makes MT more robust and can significantly
improve translation quality, even when SMT
greatly underperforms NMT. Following Marieand Fujita (2018a), our combination of NMT
and SMT works as follows.

SMT

We also trained SMT systems using Moses.
Word alignments and phrase tables weretrained on the tokenized parallel data using
mgiza. Source-to-target and target-to-source
word alignments were symmetrized with thegrow-diag-final-and heuristic. We trainedphrase-based SMT models and MSLR (monotone, swap, discontinuous-left, discontinuous910

5.1 Generation of n-best Lists
We first independently generated the 100-best
translation hypotheses with 7 NMT models,
independently trained, and also with the ensemble of these 7 NMT models. We also generated 100-best translation hypotheses with our

https://marian-nmt.github.io/, version 1.7.6
NVIDIA® Tesla® V100 32Gb.

71

Feature

Description

L2R (7)
LEX (4)
LM (2)LEN (2)

Scores given by each of the 7 left-to-right Marian models
Sentence-level translation probabilities, for both translation directions
Scores given by the language models used by the Moses baseline systems
Difference between the length of the source sentence and the length of the translation hypothesis, and its absolute value

Table 5: Set of features used by our reranking systems. The “Feature” column refers to the same featurename used in Marie and Fujita (2018a). The numbers between parentheses indicate the number of scoresin each feature set.

SMT system. We then merged all these 9 lists
generated by different systems, without removing duplicated hypotheses, which resulted
in a list of 900 diverse translation hypotheses
for each source sentence.
5.2(USMT) over unsupervised NMT (UNMT)
since previous work (Lample et al., 2018) has
shown that USMT significantly outperforms
UNMT for distant languages.
We built USMT systems using a framework
similar to the one proposed in Marie and Fujita (2018b). The first step of USMT is the induction of a phrase table from the monolingual
corpora. We first collected phrases of up to six
tokens from the monolingual corpora11 using
word2phrase.12 As phrases, we also considered all the token types in the corpora. Then,
we selected the 300k most frequent phrases in
the monolingual corpora to be used for inducing a phrase table. All possible phrase pairs
are scored, as in Marie and Fujita (2018b), using bilingual word embeddings, and the 300
target phrases with the highest scores were
kept in the phrase table for each source phrase.
As a result, the induced phrase table contains
a total of 90M (300k×300) phrase pairs. For
this induction, bilingual word embeddings of
300 dimensions were obtained using word embeddings trained with fastText13 and alignedin the same space using unsupervised Vecmap(Artetxe et al., 2018a). This alignment is themost critical step for unsupervised MT since
it is used for initializing the training. It is
expected to be extremely difficult for distantlanguages such as Myanmar, Khmer, and English, as reported by previous work (Søgaard
et al., 2018). For each phrase pair, a totalof four scores, to be used as features in thephrase tables were computed to mimic phrase-

Reranking Framework andFeatures

We rescored all the hypotheses in the list witha reranking framework using features to better model the fluency and the adequacy of eachhypothesis. This method can find a better hypothesis in these merged n-best lists than the
one-best hypothesis originated by the individual systems. We chose kb-mira as a rescoring
framework and used a subset of the features
proposed in Marie and Fujita (2018a). All thefollowing features we used are described in details by Marie and Fujita (2018a). As listed inTable 5, it includes the scores given by 7 leftto-right NMT models independently trained.
We computed sentence-level translation probabilities using the lexical translation probabilities learned by mgiza during the training of
our SMT systems. The two language models trained for SMT for each translation direction were also used. To account for hypotheses
length, we added the difference, and its absolute value, between the number of tokens in
the translation hypothesis and the source sentence.
The reranking framework was trained on nbest lists generated by decoding of the development data that we used to validate thetraining of NMT systems and to tune theweights of SMT models.

6

11
Since our Myanmar Wikipedia corpus is significantly smaller than the Myanmar Common Crawl corpus, we concatenated both corpora and used the resulting corpus in all the subsequent steps of USMT
training.
12
https://code.google.com/archive/p/
word2vec/
13
https://github.com/facebookresearch/
fastText

Unsupervised SMT

We also built an SMT system, without any
supervision, i.e., using only our monolingual
data for training. We chose unsupervised SMT
72

ID

System

my→en

en→my

km→en

en→km

1.

Moses

10.3

20.5

19.8

40.4

2.
3.
4.

Marian single
Marian single w/ backtr.
Marian ensemble of 4 w/ backtr.

15.7
19.1
22.4

25.2
28.8
29.7

17.0
24.9
25.9

37.8
42.9
43.0

5.

#1 + #4

24.8

31.3

27.5

43.9

6.

Unsupervised SMT

< 1.0

< 1.0

< 1.0

< 1.0

Table 6: Official BLEU scores for our MT systems on the official test set of the tasks. “backtr” denotes
the use of back-translated monolingual data. #5 denotes our n-best list combination described in Section
5: a combination of the best SMT and the best NMT systems realized using monolingual data. We
submitted systems #5 for human evaluation.

based SMT: forward and backward phrase andlexical translation probabilities. Finally, thephrase table and the language models wereplugged into a Moses system that was tunedon the development data using KB-MIRA.
We performed four refinement steps to improve the system, using at each step synthetic
parallel sentences generated from one thirdof the monolingual corpus, by the forward
and backward translation systems, insteadof using only either forward (Marie and Fujita, 2018b) or backward translations (Artetxe
et al., 2018b). We report on the performanceof the systems obtained after the fourth refinement step.

7

translations deliver a translation of a low quality.
Our results are more contrasted when ensembling 7 NMT models during decoding(#4). While we observe an improvement of
3.3 BLEU points for (my→en), the improvements for the other directions were limited to
1.0 BLEU points or less. Considering the costof independently training 7 NMT models andthe cost of decoding with 7 models, ensembling
does not seem to offer a cost-effective solution.
Finally, combining SMT and NMT (#5)
provides the best results with improvements
over #4 ranging from 0.9 (en→km) to 2.4
BLEU points (my→en).
Our results for unsupervised SMT (#6) follow the same trend as the results presented
by Marie et al. (2019) for English-Gujarati
and English-Kazakh at WMT19: while unsupervised MT has shown promising resultsfor European languages, it is far from being
useful for real-world applications, i.e., truly
low-resource distant language pairs. We assume that training useful bilingual weaklysupervised/unsupervised bilingual word embeddings for initializing the system remains
one of the main challenges.

Results

Table 6 presents the results for different versions of our SMT and NMT systems. We canobserve that NMT (#2) is significantly better than SMT (#1) for my-en while we canobserve the reverse for km-en. Our assumption for explaining this difference is that my-en
has a much larger training data while km-en
may not have enough to train a better NMT
systems. The extreme noisiness of the training data for km-en, that we assessed by a native Khmer speaker, may also explain the large
gap between SMT and NMT since it is wellknown that SMT is much more robust than
NMT when trained on noisy data.
Exploiting monolingual data through backtranslation (#3) consistently improves all our
NMT systems by a large margin, from 3.4(my→en) to 7.9 (km→en) BLEU points. Thishighlights the importance of using monolingual data in low-resource scenarios, even when
the NMT system used for generating back-8 Conclusion
In this paper, we showed that exploiting
cleaned and normalized noisy monolingual
data significantly helps in improving the translation quality for my-en and km-en. Furthermore, as in our previous participation in
WAT2018, we showed that combining NMT
and SMT can further improve the translation
quality over a very strong NMT system. In
order to allow participants to build state-of73

the-art MT systems, we encourage, even morethan last year, WAT organizers to provide
monolingual data for future editions of theworkshop.

Moran, Richard Zens, Chris Dyer, Ondřej Bojar,
Alexandra Constantin, and Evan Herbst. 2007.
Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th
Annual Meeting of the Association for Computational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions, pages
177–180, Prague, Czech Republic. Association
for Computational Linguistics.

References
Mikel Artetxe, Gorka Labaka, and Eneko Agirre.
2018a. A robust self-learning method for fully
unsupervised cross-lingual mappings of wordembeddings. In Proceedings of the 56th Annual
Meeting of the Association for Computational
Linguistics (Volume 1: Long Papers), pages
789–798, Melbourne, Australia. Association for
Computational Linguistics.

Guillaume Lample, Myle Ott, Alexis Conneau, Ludovic Denoyer, and Marc’Aurelio Ranzato. 2018.
Phrase-based & neural unsupervised machinetranslation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language
Processing, pages 5039–5049, Brussels, Belgium.
Association for Computational Linguistics.

Mikel Artetxe, Gorka Labaka, and Eneko Agirre.
2018b. Unsupervised statistical machine translation. In Proceedings of the 2018 Conference on
Empirical Methods in Natural Language Processing, pages 3632–3642, Brussels, Belgium. Association for Computational Linguistics.

Benjamin Marie and Atsushi Fujita. 2018a. A
smorgasbord of features to combine phrasebased and neural machine translation. In Proceedings of the 13th Conference of the Association for Machine Translation in the Americas(Volume 1: Research Papers), pages 111–124,
Boston, USA. Association for Machine Translation in the Americas.

Ondřej Bojar, Christian Federmann, Mark Fishel,
Yvette Graham, Barry Haddow, Matthias Huck,
Philipp Koehn, and Christof Monz. 2018. Findings of the 2018 conference on machine translation (WMT18). In Proceedings of the Third
Conference on Machine Translation, Volume 2:
Shared Task Papers, Brussels, Belgium. Association for Computational Linguistics.

Benjamin Marie and Atsushi Fujita. 2018b. Unsupervised neural machine translation initialized
by unsupervised statistical machine translation.
CoRR, abs/1810.12703.
Benjamin Marie, Atsushi Fujita, and Eiichiro
Sumita. 2018.
Combination of statisticaland neural machine translation for MyanmarEnglish. In Proceedings of the 32nd Pacific Asia
Conference on Language, Information and Computation: 5th Workshop on Asian Translation:
5th Workshop on Asian Translation, Hong Kong.
Association for Computational Linguistics.

Colin Cherry and George Foster. 2012. Batch tuning strategies for statistical machine translation.
In Proceedings of the 2012 Conference of theNorth American Chapter of the Association for
Computational Linguistics: Human Language
Technologies, pages 427–436, Montréal, Canada.
Association for Computational Linguistics.

Benjamin Marie, Haipeng Sun, Rui Wang, Kehai
Chen, Atsushi Fujita, Masao Utiyama, and Eiichiro Sumita. 2019. NICT’s unsupervised neural and statistical machine translation systems
for the WMT19 news translation task. In Proceedings of the Fourth Conference on MachineTranslation (Volume 2: Shared Task Papers,
Day 1), pages 294–301, Florence, Italy. Association for Computational Linguistics.

Kenneth Heafield, Ivan Pouzyrevsky, Jonathan H.
Clark, and Philipp Koehn. 2013. Scalable modified Kneser-Ney language model estimation. In
Proceedings of the 51st Annual Meeting of the
Association for Computational Linguistics (Volume 2: Short Papers), pages 690–696, Sofia,
Bulgaria. Association for Computational Linguistics.
Marcin Junczys-Dowmunt, Roman Grundkiewicz,
Tomasz Dwojak, Hieu Hoang, Kenneth Heafield,
Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev,
André F. T. Martins, and Alexandra Birch.
2018. Marian: Fast neural machine translation
in C++. In Proceedings of ACL 2018, System Demonstrations, pages 116–121, Melbourne,
Australia. Association for Computational Linguistics.

Toshiaki Nakazawa, Chenchen Ding, Raj Dabre,
Hideya Mino, Isao Goto, Win Pa Pa, Nobushige
Doi, Yusuke Oda, Anoop Kunchukuttan, Shantipriya Parida, Ondřej Bojar, and Sadao Kurohashi. 2019. Overview of the 6th workshop on
Asian translation. In Proceedings of the 6thWorkshop on Asian Translation, Hong Kong.
Association for Computational Linguistics.
Kishore Papineni, Salim Roukos, Todd Ward, andWei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In
Proceedings of 40th Annual Meeting of the Association for Computational Linguistics, pages

Philipp Koehn, Hieu Hoang, Alexandra Birch,
Chris Callison-Burch, Marcello Federico, Nicola
Bertoldi, Brooke Cowan, Wade Shen, Christine74

311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
Rico Sennrich, Barry Haddow, and Alexandra
Birch. 2016a. Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume
1: Long Papers), pages 86–96, Berlin, Germany.
Association for Computational Linguistics.
Rico Sennrich, Barry Haddow, and Alexandra
Birch. 2016b. Neural machine translation of rarewords with subword units. In Proceedings of
the 54th Annual Meeting of the Association for
Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, Berlin, Germany. Association for Computational Linguistics.
Anders Søgaard, Sebastian Ruder, and Ivan Vulić.
2018. On the limitations of unsupervised bilingual dictionary induction. In Proceedings of
the 56th Annual Meeting of the Association for
Computational Linguistics (Volume 1: Long Papers), pages 778–788, Melbourne, Australia. Association for Computational Linguistics.
Ashish Vaswani, Noam Shazeer, Niki Parmar,
Jakob Uszkoreit, Llion Jones, Aidan N. Gomez,Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the
30th Neural Information Processing Systems
Conference (NIPS), pages 5998–6008. Curran
Associates, Inc.

75