Multi-dialect Neural MachineTranslation and Dialectometry
Kaori Abe1
Yuichiroh Matsubayashi2, Naoaki Okazaki3, Kentaro Inui1,2
1. Graduate School of Information Sciences, Tohoku University
2. RIKEN Center for Advanced Intelligence Project (AIP)
3. School of Computing, Tokyo Institute of Technology
2018/12/1

1

BackgroundRecently, smart speakers(e.g. Siri) canunderstand standard languages in the world
However, (especially in Japan,) humans
sometimes use regional dialects to talkStandard
language

dialect…

gap

2018/11/30

2Motivation
Dialect translation enables smart speakers
to understand a request in a dialect
dialecttranslateStandard language

OK! I understood!

2018/11/30

3

Problem: A variety of Japanese DialectsJapanese has a large variety of dialects

Hokkaido syakkoi

Yamagata


Kochi


hiyai
Okinawa hyakkoi

Standard Japanesetsumetai(It’s cold !!)hijuru-2018/11/30

4

Problem: A variety of Japanese DialectsJapanese has a large variety of dialects→ We want smart speakers tounderstand all the dialects Yamagata

Hokkaido

syakkoiOK!

Kochi
hyakkoi



hiyai

OK!

Okinawahijuru-OK!
OK!

2018/11/30

5

Problem: Lack of Dialect Corpus
Japanese Dialect Corpus:
48 dialects  30 minutes dialog→ 34,117 sentence pairs of Transcript(718 sentence pairs per dialect)

Dialects are spoken
rather than written

It’s too small !!!

2018/12/1

6

Problem: Lack of Dialect Corpus
Japanese Dialect Corpus:
48 dialects  30 minutes dialog→ 34,117 sentence pairs of Transcript(718 sentence pairs per dialect)

Dialects are spoken
rather than written

It’s too small !!!
We should consider how to make the best use ofthis small language resource

2018/12/1

7

Relation between
a Japanese Dialect and Standard JapaneseFrom a Japanese dialect to standard Japanese,Change : vocabulary and particular syllables
Unchange : word order
Example : My mother said “It’s cold !”Aomori Dialect
Standard Japaneseoga “ ” ga
to“syakkoi !” “ ”haha ga “tsumetai !”My mother

2018/12/1“It’s cold”

yutta 
ittatosaid8

Relation between
a Japanese Dialect and Standard JapaneseFrom a Japanese dialect to standard Japanese,Change : vocabulary and particular syllables
Unchange : word order
Example : My mother said “It’s cold !”Aomori Dialect
Standard Japaneseoga “ ” ga
to“syakkoi !” “ ”haha ga “tsumetai !”My mother

2018/12/1

In our study, we use a phonetic lettersystem named “kana” (not “kanji”)“It’s cold”

yutta 
ittatosaid9Relation between
a Japanese Dialect and Standard JapaneseFrom a Japanese dialect to standard Japanese,Change : vocabulary and particular syllables
Unchange : word order
Example : My mother said “It’s cold !”Aomori Dialect
Standard Japaneseoga “ ” ga
to“syakkoi !” “ ”haha ga “tsumetai !”My mother

2018/12/1

In our study, we use a phonetic lettersystem named “kana” (not “kanji”)“It’s cold”

yutta itta
tosaid10

Relation between
a Japanese Dialect and Standard JapaneseFrom a Japanese dialect to standard Japanese,Change : vocabulary and particular syllables
Unchange : word order
Example : My mother said “It’s cold !”Aomori Dialect
Standard Japaneseoga “ ” ga
to“syakkoi !” “ ”haha ga “tsumetai !”My mother2018/12/1

In our study, we use a phonetic lettersystem named “kana” (not “kanji”)“It’s cold”

yutta itta
tosaid11

Relation between
a Japanese Dialect and Standard JapaneseFrom a Japanese dialect to standard Japanese,Change : vocabulary and particular syllables
Unchange : word order
Example : My mother said “It’s cold !”Aomori Dialect
Standard Japaneseoga “ ” ga
to“syakkoi !” “ ”haha ga “tsumetai !”My mother

2018/12/1

In our study, we use a phonetic lettersystem named “kana” (not “kanji”)“It’s cold”

yutta itta
tosaid12Model (Fixed-order + Character)Fixed-order + Character NMT
Hokkaido dialect
Input“It’s cold”
said  Dialect NMTtoAttentiontos
2018/11/30

yayu

Output



tk



kota


i

tsui




met




tata






i



13

Japanese DialectsJapanese dialects have similarity tosome extent

Hokkaido syakkoi

Yamagata


Kochi


hiyai

hyakkoi

Standard Japanesetsumetai(It’s cold !!)hijuru-2018/11/30

14

Similarity between Japanese Dialects
Most of the dialects share fundamental properties
Same or Similar Vocabulary
Hokkaido dialectsyakkoiSame Vocabulary

Yamagata dialectSimilar VocabularySame VocabularyStandard Japanese
2018/11/30

This water

hyakkoi

Different Vocabulary


is
cold!

15

Similarity between Japanese Dialects
Most of the dialects share fundamental properties
Phonetic correspondence rules
Hokkaido dialect
Standard Japaneseimawa


soudewa

naigedonaimawa
Nowsoudewa
it isnaikedonaSame rule

Yamagata dialectsakana


ga


yagedaStandard JapanesesakanaFishga

yaketa2018/11/30

not


grilled16

Approach (Multilingual NMT)A multilingual NMT[Johnson+, 2017] utilizes sharedproperties between dialects by way of a unified modelthat learns multiple languages jointly
Hokkaido

Yamagata

Kochi

2018/11/30

syakkoi

hyakkoi




hiyai

MultilingualNMT

Automaticallylearning similarity
betweenlanguages

StandardJapanesetsumetai(cold)

17

Model (Character + Fixed-order + Multilingual)Character-level + Fixed-order + Multilingual NMT
Hokkaido dialect

Input“It’s cold”
 saidMulti-Dialect NMTtsu

Attention<Hokkaido> s
2018/11/30

yaOutputkko <standard>imetai

18Experiments

2018/11/30

19

Setting
Corpus :48 dialects  30 minutes dialog→ 34,117 sentence pairs (116,928 “bunsetsu” pairs)(718 sentence, 2436 “bunsetsu” pairs per dialect)
Train : Validation : Test = 8 : 1 : 1

Evaluation : BLEU (character-level)
SystemNMT : OpenNMT-pySMT (baseline system) : Moses
2018/11/30

20

Evaluation
Our model archived the best performance
SystemCharact
er-level

Fixedorder

Dialect jointly BLEU
labels learning
35.10

Original
Multi NMT





75.66

Mono NMT







22.45

Sentence-Multi NMT







71.29

Multi NMT (w/o labels)







69.74

Mono SMT









52.98

Multi SMT (w/o labels)









73.54

2018/12/1

21

Mono NMT vs. Multi NMT
SystemCharact
er-level

Fixedorder

Dialect jointly BLEU
labels learning
35.10

Original
Multi NMT







75.66

Mono NMT





22.45

Sentence-Multi NMT





71.29

Multi NMT (w/o labels)







69.74

Mono SMT







52.98

 better l Multilingual NMT
performs
significantlythanmonolingual NMT

Multi SMT (w/o labels)
2018/12/1

73.54

22

Original vs. Mono NMT
SystemCharact
er-level

Fixedorder

Dialect jointly BLEU
labels learning
35.10

Original
Multi NMT







75.66

Mono NMT









22.45

Sentence-Multi NMT







71.29

Multi NMT (w/o labels)







69.74

Mono SMT

52.98Multi
SMT (w/o translation
 becauseeach 73.54
lMonoNMTlabels)
could not learn
rulesmonolingual dialect-to-standard corpus is too small
2018/12/123

Sentence vs. Fixed-order
SystemCharact
er-level

Fixedorder

Dialect jointly BLEU
labels learning
35.10

Original
Multi NMT







75.66

Mono NMT







22.45

Sentence-Multi NMT







71.29

69.74




l Although
it has a weak
MonoSMT
52.98
 point that it cannot
 considercontext, the strategy of fixed-order translation is effective
Multi SMT (w/o labels)
73.54




Multi NMT (w/o labels)

2018/12/1

24

With labels vs. Without labels
SystemCharact
er-level

Fixedorder

Dialect jointly BLEUlabels learning
35.10

Original
Multi NMT







75.66

Mono NMT







22.45

Sentence-Multi NMT







71.29

Multi NMT (w/o labels)







69.74

Mono SMT







52.98

73.54



l Dialect labels contribute to increasing the BLEU scorebecause it clearly teach the NMT what the input dialect isMulti SMT (w/o labels)
2018/12/1

25

SMT vs. NMT
SystemCharact
er-level

Fixedorder

Dialect jointly BLEU
labels learning
35.10

OriginalMulti NMT







75.66

Mono NMT







22.45

Sentence-Multi
NMT outperformedl Our model
standardSMT models!


71.29

Multi NMT (w/o labels)









69.74

Mono SMT







52.98

Multi SMT (w/o labels)







73.54

2018/12/1

26

Effectiveness of Dialect labels
SimilarCompared with standard Japanese

Different0With dialect labels

Without dialect labels

0Dialect labels which teach what the dialect is contributeto increasing BLEU scores in most of the dialects
2018/11/30

27

Effectiveness of Dialect labels
SimilarCompared with standard JapaneseDifferent

0With dialect labels

Without dialect labels

0

Our model could not much perform in Okinawa dialect because
it is quite different from standard Japanese
2018/11/30

28

Analysis

2018/11/30

29

Effect of Nearby Dialects
Assumption: the data of nearby dialects might contribute to
the high performance under the multilingual architecture
Setting

For ”Gifu” dialect …
Data1: Removed the nearest 5 dialects
Data2: Removed the farthest 5 dialects
2018/11/30

30

Analysis (Effect of Nearby Dialects)
Evaluating whether neighbor dialect data improves
a BLEU score
DatasetAvg. Δ #Regions BLEU
decreased
All -nearest 5 -0.94 34/48(71%)
All -farthest 5 -0.22 31/48(65%)
The data of near areas are more effective
for multilingual NMT
The lack of 5 dialects in supervision data affecttranslation accuracy in a low-resource setting
2018/12/1

31

Analysis (Visualize dialect embeddings)
A t-SNE projection of dialect embeddings follows
dialectological typology
The nearer the distance between two areas is, the more similar
dialects are used (background colors)

2018/11/30

32

Analysis (Visualize dialect embeddings)
A t-SNE projection of dialect embeddings follows
dialectological typology
The nearer the distance between two areas is, the more similar
dialects are used (background colors)

2018/12/1

33

Japanese dialectal typology
Dialectological researcher said that dialects spreadfrom an ancient capital city to remote areas
concentrically [Yanagida, 1980]

2018/12/1

34

Analysis (Visualize dialect embeddings)
A t-SNE projection of dialect embeddings followsdialectological typology
Though the distance between D and E is far away,
similar dialects are used

2018/12/1

35

Conclusions
 We presented Multi-dialect NMT systemcharacter-level + fixed-order + multilingualThe unified model that learns similar multiple
dialects jointly is effective for multi-dialect
translation
We can observe similar relationships to the existing
dialect typology in some dialects by analyzing
similarity of the dialect embeddings

2018/11/30

36

References
[Johnson+, 2017] Google’s Multilingual Neural
Machine Translation System: Enabling Zero-Shot
Translation, ACL2017
[Yanagida, 1980] Kunio Yanagida. 1980. “Kagyuko”.
Iwanami Shoten, Publishers.

2018/12/1

37

Appendix

2018/11/30

38

FAQ
Can you construct standard-to-dialect Multi NMTwith the same model?
Yes (But the translation accuracy dropped)
Due to a weak language model in each target dialectWhy is there not a “Multi SMT (w/ label)” setting ?
We could not devise an alternative to dialect labels inSMT

2018/11/30

39

Is Word Order really unchanged?
 We checked 100 dialect-standard sentence pairs inall 48 dialects→ All the pairs are unchanged

2018/11/30

40

The Sentence Pairs of each dialect

2018/11/30

41

Analysis (Translation Examples: Good)The output of Multi NMT completely agree with the reference
Example : “(We) had skated, aren’t we?”
Source(Aomori dialect)

shi ke - to / no ri su ta de ba  /  

Referencesu ke - to / no tsu ta de ha na i de su ka /   Multi NMT

su ke - to / no tsu ta de ha na i de su ka
 /   Sentence-Multi NMTu ke i to / no ri shi ta de ha na i de su ka
 /    

Multi NMT(w/o labels)

shi ke - to / no tsu ta de ha na i de su ka
 /   

Multi SMT(w/o labels)

su ke - to / no tsu ta de ha na i de su /   

2018/11/30

42

Analysis (Translation Examples: Bad)
Multi NMT could not translate too rare word
Example : “(I) ran a horse”
Source(Okinawa dialct)

ma - / pa ra - chi ya  / Referenceu ma / ha shi ra se te ne / Multi NMTu ma / ha ra de ha / 

Sentence-Multi NMTa a / ha ra u shi ya
 / Multi NMT(w/o labels)

ma a / ha na shi te ha
/Multi SMT(w/o labels)

ma a / pa ra - to ne / 

2018/11/30

43