言語処理学会 第30回年次大会 発表論文集（2024年3月）

Robust Neural Machine Translation for Abugidas
by Glyph Perturbation
Hour Kaing1 Chenchen Ding1 Haiyue Song1 Jiannan Mao21 Hideki Tanaka1 Masao Utiyama1
1 ASTREC, UCRI, NICT, Japan 2 Gifu University, Gifu, Japan
1 {hour_kaing, chenchen.ding, haiyue.song, hideki.tanaka, mutiyama}@nict.go.jp
2 mao@mat.info.gifu-u.ac.jp

Abstract

Glyph:

Neural machine translation (NMT) systems are vulnerable when trained on limited data. This is a common scenario in low-resource tasks in the real world. To increase
robustness, researchers intently added realistic noise in the
training phase. Noise simulation using text perturbation
has been proven to be efficient in writing systems that use
Latin letters. In this study, we further explore perturbation
techniques on more complex abugida writing systems, forwhich the visual similarity of complex glyphs is considered
to capture the essential nature of these writing systems. Besides the generated noise, we investigated three training approaches such as subword regularization, adversarial training, and consistency training. Finally, we propose to combine them to maximize the translation performance. We
conducted experiments on six languages: Bengali, Hindi,
Myanmar, Khmer, Lao, and Thai. Our training approachobtained the best performance for five languages.

1

Introduction

Neural machine translation (NMT) systems have beenshown to be vulnerable in noisy settings, where slightly
modified inputs cause serious translation failures [1, 4].
Boucher et al. [2] showed that techniques using pre-trained
language models cannot prevent this. This drawback
is more disastrous in low-resource scenarios, where themodel’s robustness becomes a crucial issue.
Several text perturbation techniques have been developed to improve robustness by introducing synthesized textual noise [12, 6]. Most techniques mostly focus on languages that use alphabetic systems, such as Latin letters.
As a more complex writing system, Chinese characters
were investigated by Zhang et al. [20]. In the present study,য়ুBengali

Header: U+ 09 **Original: AF BC C1
Perturbed: AF BE C1 C1 ①
Perturbed: AF C1 BC ②Glyph:

ड़ेHindi

စွဲ Myanmar

U+ 09 **
U+ 10 **
21 3C 47
05 3D 32
21 3C 47 47 ① 05 3D 32 32 ①
21 47 3C 47 ① 05 39 1D 32 ③េធ្វ ី Khmer ຫຼ# Lao

ผู้

Thai

U+ 0E **
U+ 0E **Header: U+ 17 **
AB
BC
CD
C8
1C
39 49
92
D2
9C
BE
Original:
Perturbed: 92 D2 9C C1 B8 ③ AB CD C8 BC ② 1C 49 39 ②
AB C8 CD BC ②
Perturbed: 92 BE D2 9C ②

Figure 1 Homoglyph perturbation examples for various
abugida systems. The Unicode of each character is listed below
the glyph. Perturbed characters are emphasized in bold font. Various patterns cause homoglyphs: 1) repetition, 2) permutation,and 3) decomposition (e.g., BE → C1 B8 in Khmer).

we further fill the gap in text perturbation techniques forunderstudied abugida writing systems, which vary and areused widely in South-East Asia.
A reasonable perturbation technique should produce
meaningful and readable text that is indistinguishable forhumans, but disastrous for a system’s prediction [11]. Visually similar glyphs or homoglyphs1） were investigatedin Eger et al. [6], Boucher et al. [2], and Le et al. [11] obtained realistic samples from large corpora. As a primarycontribution, we further develop these previous studies forabugida writing systems. Some exemplary homoglyphs invarious abugida systems are illustrated in Figure 1.
To address noise, we propose a training strategy thatleverages adversarial training, subword regularization, andconsistency training. We selected six languages that use
abugida systems, Bengali, Hindi, Myanmar, Khmer, Lao,and Thai, and experimented on them for low-resource
tasks. Overcoming noisy perturbations improved the robustness, with non-degenerate performance.
1） I.e., glyphs with identical visuals, but different encodings.

― 560 ―

This work is licensed by the author(s) under CC BY 4.0(https://creativecommons.org/licenses/by/4.0/).

2

Background: Abugidas

𝑍 (𝑥𝑖 ) =Õ

(2)𝑠𝑐𝑜𝑟𝑒(𝑥 00 , 𝑥 𝑖 , 𝛽)𝑥 00 ∈𝑉\{ 𝑥𝑖 }An abugida is a writing system that combines featuresof both syllabic and segmental systems. Text is written𝑠𝑐𝑜𝑟𝑒(𝑎, 𝑏, 𝛽) = 𝐼 (𝑠(𝑎, 𝑏) ≥ 𝛽) · 𝑠(𝑎, 𝑏),(3)

as a sequence of syllables, which resemble Japanese hira-where 𝐼 (·) is an indicator function; 𝛼 and 𝛽 control thegana, but can be broken down into separate consonants andvowels, as in a segmental system. A typical abugida sylla-

respectively; and 𝑠(𝑎, 𝑏) is a similarity function between

ble consists of a base consonant accompanied by a default
vowel or additional vowels represented by diacritics. In
computer systems, these syllables are rendered into glyphs,
which are visual symbols in the rendering process. A glyph
represents a letter or a composition of multiple letters. For
example, in Latin, the letter a is a glyph, and combinedwith a grave accent (diacritic), it becomes another glyph à;
similarly, in abugidas, as shown in Figure 2, a consonant ischance of 𝑥𝑖 being perturbed and the similarity threshold,the continuous vectors of two tokens 𝑎 and 𝑏, for example, the cosine similarity 𝑠(𝑎, 𝑏) = cos(𝑣(𝑎), 𝑣(𝑏)), andwhere 𝑣(·) is a vector. The overall perturbation process is
illustrated in Figure 3. Next, we present the process for
obtaining 𝑉 from corpora that contain diverse adversarial
candidates in Section 3.1.2, and describe how vector 𝑣(·) isrepresented by an image in Section 3.1.3 and by counting
diacritics in Section 3.1.4.

represented by a glyph, as in (a), and combined with multiple diacritics to become another glyph, as in (b). As in
Figure 1, similar glyphs or homoglyphs commonly occur

replaceand glyph token to refer to its Unicode characters.

ធ

(a)

ធ
(b)

េធ្វ ី

ធ◌្វេ◌ី

sample𝑥!$ 𝑥#$ … ∈ 𝑉

Figure 3

merous patterns and are difficult to engineer. We explore
Hereafter, we use the term glyph to refer to a visual symbol

weights
0.1 0.5 …

𝑥! … 𝑥" …

in the composition of complex diacritics, which have nusuch diacritic composition from human-generated corpora.

𝑥′

3.1.2

𝑔(𝑥 $ |𝑥" )Overall perturbation processing.

Vocabulary Construction

This step is similar to a typical vocabulary preparation
process that consists of tokenization and unique token ex-

Left Right

Left: a glyph
Right: Unicode characters: diacritics

traction to obtain 𝑉. Specifically, we categorize each character as a consonant or diacritic based on Unicode Standard
data. After that, we base tokenization on the consonant po-

Figure 2 Examples of Khmer glyphs. (a) is a glyph without
diacritics and (b) with diacritics.

sition such that each token starts with a consonant followed3

Proposed Method

tokens as 𝑉. Additionally, because the similarity is mostly

3.1

Perturbation for Abugidas

3.1.1

Overall Processing

around the diacritics, we want to perturb only the diacritic
parts of each targeted token. To achieve this, our trick is toreplace the consonant counterpart of each token in 𝑉 withthat of the targeted token 𝑥𝑖 , which varies every time step 𝑖.

Given a sentence x = (𝑥1 , . . . , 𝑥 𝑛 ), each token 𝑥 𝑖 has achance of being replaced with an adversarial candidate 𝑥 0 ∈𝑉 chosen based on its similarity score w.r.t 𝑥 𝑖 [6], where 𝑉
is vocabulary that contains all possible tokens, including
clean and noisy tokens2）. The perturbation probability foreach targeted token 𝑥𝑖 is formulated as𝑠𝑐𝑜𝑟𝑒(𝑥 0 , 𝑥 𝑖 , 𝛽)


𝛼
·
,


𝑍 (𝑥𝑖 )
𝑔(𝑥 0 |𝑥𝑖 ) = 1 − 𝛼,by many or zero diacritics, and then obtain a list of uniqueThis trick is based on the assumption that the visual formof the consonant never changes when it is combined withdiacritics. However, we discovered one case in Bengali and
Hindi in which the base consonant changed its visual form.
Hence, we simply skipped the perturbation for such case.
3.1.3

if 𝑥 0 ≠ 𝑥𝑖
otherwise2） As 𝑉 is fixed in practice, we skip the process if 𝑥𝑖 ∉ 𝑉.

Image-based Glyph Embeddings (IGE)

We convert each glyph image3）into a linear vector of(1)

𝑚 · 𝑛 dimensions by arranging rows in the 𝑚 × 𝑛 matrix,3） We used Pillow9.4.0 and Google Noto Serif fonts with 100px for
all languages.

― 561 ―

This work is licensed by the author(s) under CC BY 4.0(https://creativecommons.org/licenses/by/4.0/).

where each entry corresponds to a pixel in the grayscale im-similar text exists in the training data and can be used inage. The pixel values range from 0 (representing the empty

our robust training. In this study, we use unigram subword

area) to 255 (representing the visible part of a glyph). Be-

tokenizer [10], and sample subwords with the 𝑛-best size

cause the image size varies greatly across glyphs, we pre-𝑙 = ∞ and distribution smoothness 𝜇 = 0.2.

determine the maximum size 𝑚 × 𝑛 based on all glyphs andWe further adopt consistency training [17] to enforce thethen render them into the 𝑚 ×𝑛 size. They must align to theleft on the horizontal axis and to the middle on the verti-

model’s prediction being invariant between noisy and clean
inputs. Let x is an input sequence and x0 is its noisy variant.

cal axis. Additionally, we empty the pixels that correspond𝑛 , the objective
Therefore, given a training set {(x𝑖 , y𝑖 )}𝑖=1

to the consonant to ensure that the similarity value is not

function can be expressed asÕ 1
1L(𝜃) =[− log 𝑝 𝜃 (y𝑖 |x𝑖 ) − log 𝑝 𝜃 (y𝑖 |x𝑖0 )2
2+ 𝜆𝐷 ( 𝑝 𝜃 (y𝑖 |x𝑖 )|| 𝑝 𝜃 (y𝑖 |x𝑖0 ))],

affected by the common pixels of the base consonant. Finally, we use the cosine similarity function for IGE, which
is defined as 𝑠(𝑎, 𝑏) = cos(𝑣(𝑎), 𝑣(𝑏)).
3.1.4

where 𝜃 is a set of model parameters and 𝐷 (·||·) is a non-Diacritic-Count Embeddings (DGE)

A simpler approach involves counting the diacritics thatexist in a glyph token and how many times they occur.
Specifically, a glyph token is represented by a frequency
vector, where each entry corresponds to a diacritic in thelanguage and the value of each entry is the count of thecorresponding diacritic in the glyph token. Additionally,
we smooth each frequency value using an exponent 𝛾. For
instance, if a language 𝑙 has a set of diacritics {𝑎, 𝑏, 𝑐}and a glyph token consists of diacritics 𝑎𝑐𝑐, DGE represents it using a frequency vector [1, 0, 2] 𝛾 because 𝑎 occurs once and 𝑐 occurs twice. Using DGE, we can identify two glyphs that have similar sets of diacritics, regardless of the order of the diacritics. We set 𝛾 = 0.3
in all experiments and use the inverse Euclidean distanceas the similarity function, which is defined as 𝑠(𝑎, 𝑏) =(𝐸𝑢𝑐𝑙𝑖𝑑𝑒𝑎𝑛(𝑣(𝑎), 𝑣(𝑏)) + 1) −1 .

3.2

(4)

negative distance metric between two distributions that arecontrolled by the hyperparameter 𝜆. Following Wang et al.
[17], we use Kullback–Leibler divergence for 𝐷 (·||·) andset 𝜆 = 0.2.

4

Experiments

4.1

Settings

We experimented on six abugida languages: Bengali(bg), Hindi (hi), Myanmar (my), Khmer (km), Lao (lo),and Thai (th). We constructed 𝑉 from the cleaned CommonCrawls [18, 3] and evaluated translation performanceon the Asian Language Treebank dataset [16] from abugida
languages to English. We tokenized the training/validation/test sets using SentencePiece, with a joint vocabularyof 4k. We perturbed the test data using IGE and the training
data using either IGE or DGE.

Robust NMT Training

We used the transformer architecture for all the models

To generalize a model in the presence of noisy inputs, we
explore variable noises and train the model on them. Previously, Eger et al. [6] proposed adversarial training (AT)that perturbs the training data in the same manner as thetest data, while Kudo [10] introduced subword regularization (SR), which samples variable subwords for training.
Both techniques have been proven effective against noisy
inputs. In this work, we combine both techniques by first
perturbing each training sample and then sampling variable subwords for each perturbed sample. For generalization, we extract 𝑉 to perturb the test data from external corpora, whereas we extract 𝑉 to perturb the training data only
from the training data itself. We hypothesize that visuallyand implemented them using Fairseq [13] in our experiments. We trained all the models on the eight GPUs (Tesla
V100 SXM2 with 32 GB memory) and the number of parameters was approximately 54 million. We mostly based
the configuration on Guzmán et al. [7], which was specifically designed for the Indic low-resource NMT setting.
However, we further fine-tuned the number of epochs andfound that increasing the number of epochs to 1k achieved
improvements across all models.

4.2

Results and Discussions

We evaluated the performance of the vanilla model(Base) with respect to our perturbation technique using
𝛽 = 1. The results are presented in Figure 4. We ob-― 562 ―

This work is licensed by the author(s) under CC BY 4.0(https://creativecommons.org/licenses/by/4.0/).

BLEU
BLEU

30
20
10
0
30
20
10
0

Bengali

Hindi

Khmer

0.0

Myanmar

Lao

1.0 0.0

Table 1 BLEU results on perturbed inputs. SR𝑐𝑡 denotes SR
with consistency training. ∗ and † indicate the significance values
𝑝 < 0.01 and 𝑝 < 0.05 compared with SR𝑐𝑡 , respectively.

Thai

1.0 0.0

1.0

bg

hi

my

km

lo

th

Base

13.8

8.2

6.8

12.0

14.7

19.5

SR

16.6

11.0

15.3

17.9

16.8

20.6

SR𝑐𝑡

18.8

14.0

17.2

19.5

18.7

22.5

IGE

18.1

14.8∗

18.3∗

21.1∗

19.1†

22.5

DGE

19.0∗

14.4†

18.3∗

21.2∗

18.7

22.5

Figure 4 BLEU scores of NMT with 𝛼 from 0.0 to 1.0, with astep of 0.2 on the x-axis, and 𝛽 set to 1.0.

[12, 5], whereas in the black-box scenario, only the model’s

served performance degradation across all languages, with

operations have been proposed, such as randomly insert-a more significant influence observed for Hindi, Myanmar,
and Khmer. This could be attributed to the higher number of perturbed glyphs in these languages compared with
Bengali, Lao, and Thai. The details regarding the numberof perturbed glyphs per line are presented in Figure 5.
Table 1 demonstrates the effectiveness of the robust approaches against perturbation, even with SR, and show the
improved performance of SR with consistency training,
SR𝑐𝑡 . Our robust training using IGE and DGE further improved robustness performance, in most cases, while maintaining comparable performance on clean input, as discussed in the Appendix A. It is noteworthy that our approach only exploited the visual similarity within the train-

input and output are known [12, 5]. Various perturbation
ing/deleting/replacing/swapping characters [9], character
shuffling, perturbation based on the keyboard layout andnatural typos [1], extraction of visually similar glyphs ofcharacters [6], and similar embedding subwords [14]. Our
study is similar to that of Eger et al. [6], where visuallysimilar glyphs were explored.

5.2

Consistency Training

In various studies, researchers have used consistency
training in various ways to enhance the performance of
natural language processing (NLP) models. Previously,
Wang et al. [17] used consistency training to improve subword tokenization in multilingual models. Xie et al. [19]ing set and the results supported our hypothesis.

and Kambhatla et al. [8] improved data augmentation tech-

In this study, we only examined the impact of perturbation using homoglyphs with 𝛽 = 1. Reducing 𝛽, for exam-

Park et al. [15] used consistency training on virtual noiseple, to 0.95, would be likely to result in a decrease in NMT
performance. However, this may not be a realistic scenario
because it would also reduce the readability of the text forhumans. We provide examples of perturbed samples withniques for NMT using consistency training. Furthermore,
to improve the performance of text classification and natural language inference tasks. In this study, we adopted
consistency training to regularize our training on the joint
sampling of adversarial text and subwords to enhance thesmaller 𝛽 in Table 3 in the Appendix. Interestingly, a nativerobustness of the NMT model against perturbations.

Khmer speaker was able to read samples with a 𝛽 value of

6

0.9, which indicates that the perturbed text was still readable at lower 𝛽 values. However, more extensive assessments with native speakers are required in our future study

Conclusion
In this study, we presented a perturbation approachthat leverages visual similarity and introduced a training

to better understand the impact of 𝛽 on text readability.

strategy to maintain the performance of the NMT model.

5

Related Workthrough experiments that perturbed test data using mono-

5.1

Text Perturbation

We exposed the vulnerability of the vanilla NMT model
glyphs, and demonstrated the importance of robust training
against text perturbation. The findings of this study can

Text perturbation has been extensively studied in the literature, with two scenarios: white-box and black-box. In
the white-box scenario, the model’s gradients are leveraged

aid future research effort in evaluating the generalization
capabilities of NMT models, particularly for low-resource
settings and understudied languages.

― 563 ―

This work is licensed by the author(s) under CC BY 4.0(https://creativecommons.org/licenses/by/4.0/).

References

Dongwon Lee. Perturbations in the wild: Leverag-[1]Yonatan Belinkov and Yonatan Bisk. Synthetic andversarial attack and defense. In Proc. of ACL, pages

ing human-written text perturbations for realistic ad-

natural noise both break neural machine translation. In
Proc. of ICLR, 2018.

2953–2965, 2022.
[12]Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting

[2]Nicholas Boucher, Ilia Shumailov, Ross Anderson, andNicolas Papernot. Bad characters: Imperceptible nlp

Wang. Textbugger: Generating adversarial text against
real-world applications. In Proc. of NDSS, 2018.

attacks. In Proc. of SP), pages 1987–2004. IEEE,[13]Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan,

2022.

Sam Gross, Nathan Ng, David Grangier, and Michael[3]Alexis Conneau, Kartikay Khandelwal, Naman Goyal,

Auli. fairseq: A fast, extensible toolkit for sequencemodeling. In Proc. of NAACL-HLT, 2019.

Vishrav Chaudhary, Guillaume Wenzek, Francisco
Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer,[14]Jungsoo Park, Mujeen Sung, Jinhyuk Lee, and Jaewoo

and Veselin Stoyanov. Unsupervised cross-lingual rep-Kang. Adversarial subword regularization for robust

resentation learning at scale. In Proc. of ACL, pages

neural machine translation. In Proc. of EMNLP,

8440–8451, 2020.

2020.

[4]Javid Ebrahimi, Daniel Lowd, and Dejing Dou. On ad-[15]Jungsoo Park, Gyuwan Kim, and Jaewoo Kang. Con-versarial examples for character-level neural machinesistency training with virtual adversarial discrete per-translation. In Proc. of COLING, 2018.

turbation. In Proc. of NAACL, pages 5646–5656,[5]Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing
Dou. Hotflip: White-box adversarial examples for text

2022.
[16]Hammam Riza, Michael Purwoadi, Gunarso, Teduhclassification. In Proc. of ACL, 2018.

Uliniansyah, Aw Ai Ti, Sharifah Mahani Aljunied, Lu-[6]Steffen Eger, Gözde Gül Şahin, Andreas Rücklé, Ji-

ong Chi Mai, Vu Tat Thang, Nguyen Phuong Thai,

Ung Lee, Claudia Schulz, Mohsen Mesgar, Krishnkant

Rapid Sun, Vichet Chea, Sethserey Sam, Sopheap

Swarnkar, Edwin Simpson, and Iryna Gurevych. Text

Seng, Khin Mar Soe, Khin Thandar Nwet, Masao

processing like humans do: Visually attacking andUtiyama, and Chenchen Ding.

shielding nlp systems. In Proc. of NAACL, pages

Asian language treebank. In Proc. of O-COCOSDA,

1634–1647, 2019.

pages 1–6, 2016.

[7]Francisco Guzmán, Peng-Jen Chen, Myle Ott, Juan

Introduction of the[17]Xinyi Wang, Sebastian Ruder, and Graham Neubig.

Pino, Guillaume Lample, Philipp Koehn, Vishrav

Multi-view subword regularization.

Chaudhary, and Marc’Aurelio Ranzato.

NAACL, pages 473–482, 2021.

The flores

evaluation datasets for low-resource machine translation: Nepali-english and sinhala-english. In Proc. of

In Proc. of[18]Guillaume Wenzek, Marie-Anne Lachaux, Alexis
Conneau, Vishrav Chaudhary, Francisco Guzmán, Ar-

EMNLP, page 6098â��6111, 2019.

mand Joulin, and Edouard Grave. Ccnet: Extract-[8]Nishant Kambhatla, Logan Born, and Anoop Sarkar.

ing high quality monolingual datasets from web crawl

Cipherdaug: Ciphertext based data augmentation fordata. In Proc. of LREC, pages 4003–4012, 2020.

neural machine translation. In Proc. of NAACL,[19]Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong,

pages 201–â��218, 2022.

and Quoc Le. Unsupervised data augmentation for[9]Vladimir Karpukhin, Omer Levy, Jacob Eisenstein,
and Marjan Ghazvininejad.

consistency training.

Training on synthetic

noise improves robustness to natural noise in machine

In Proc. of NeurIPS, vol-ume 33, pages 6256–6268, 2020.
[20]Zihan Zhang, Mingxuan Liu, Chao Zhang, Yimingtranslation. In Proc. of W-NUT, 2019.

Zhang, Zhou Li, Qi Li, Haixin Duan, and Donghong[10]Taku Kudo. Subword regularization: Improving neural network translation models with multiple subword

Sun. Argot: generating adversarial readable chinese
texts. In Proc. of IJCAI, pages 2533–2539, 2021.

candidates. In Proc. of ACL, pages 66–75, 2018.
[11]Thai Le, Jooyoung Lee, Kevin Yen, Yifan Hu, and― 564 ―

This work is licensed by the author(s) under CC BY 4.0(https://creativecommons.org/licenses/by/4.0/).

5/17/23, 3:02 PM

noisy_samples.html

clean: េមេ គេនះ5/17/23, 3:02 PMយឆងែតមន
។់ នុ ស េឡីយnoisy_samples.html
ិ ឆងដលម

β=1:
clean:
េមេ គេនះ យឆងែតមន់ នុ ស េឡីយ។ិ ឆងដលម
β=.95: េមេេមេ់ គេនះ ៑ យឆងយឆងែតមឆួ ងដលមេ៍ ឡីឺ យ។
β=1:
់ ែតម
់ េឡ
៎ ន
់ នុ សិ ឆងដល
β=.9: េមែមេ់៎ គេនះគេន៍ះ ៑៎ យឆ
យឆ៍ង
ឆងដល
ង
β=.95:់ ដលម៍ ឡឺ់ យ។់ ែតម់ នុ ស់៑ ុ េេឡ
៎៑ ន
ិី ឆួ

A

Additional Results and Analysisβ=.9: ែមេ៎ គេន៍ះ ៎ យឆ៍ងែតម៑ នី ឆង់ ដលម់ នុ ស៑ ុ េឡ់យ។

clean: ໄຂ້ຫວັດໃຫຍ່ແມ່ນ ພະຍາດຕິດຕຂະໜາດສູງ ແຕ່ບສາມາດຕິດຕຫາຄົນໄດ້.

The impact of our perturbation technique with 𝛽 = clean:
1
ໄຂ້
β=1:
ໄຂ້ຫ
ຫວັ
ວັດ
ດໃຫຍ່
ໃຫຍ່ແ
ແມ່
ມ່ນ
ນ ພະຍາດຕິ
ພະຍາດຕິດ
ດຕຂະໜາດສູ
ຕຂະໜາດສູງ
ງ ແຕ່
ແຕ່ບ
ບສາມາດຕິ
ສາມາດຕິດ
ດຕຫາຄົ
ຕຫາຄົນ
ນໄດ້ໄດ້..
5/17/23, 3:02 PM

noisy_samples.html

on the dataset is measured by the amount of noise introβ=1:ໃຫຍ່ແດ
β=.95: ໄຂ້
ໄຂ້ຫ
ຫວັ
ວັດ
ດໃຫຍ່
ແມ່
ມ່ນ
ນ ພະຍາດຕິພ່ ະຍ໋າດຕິດ
ດຕຂະໜາດສູ
ຕຂະໍໜາດສູ
່
່ງ
ງ ແຕ່
ແຕ່ບ
ບສາມາດຕິ
ສາໍມາດຕິ
່ດຕຫາຄົຕຫ໋ າຄ່ນ
ົນໄດ້
ໄດ້..
clean: េមេ គេនះយឆងែតមនិ ឆងដល់មនុ ស េឡីយ។េមេ គេនះ

យឆងែតមនិ ឆងដល់មនុ ស េឡីយ។

β=1:

β=.95:
ໄຂ້
ວັ
ແ
າາດຕິ
່ ឺ យ។ ູ່ງ
duced, as shown in Figure 5. The graph indicates thatβ=.95: េម់មនុ សេ៍ ឡາດສູ
៑ យឆ់
់ ະໜ
៎ ន
ិະ
β=.9:
ໄຂ້េຫ
ຫគេនះ
ວັດໃຫຍ່
ດໃຫຍ່
ແມ່
ມ່ងນ
ນែតພ່
ພ່
ະឆួຍ໋
ຍ່ងដល់
ດຕຶດດមຕຂະໍ
ຕຂື
ໜາດສູ
ງ ແຕ່
ແຕ່ບ
ບສາໍ
ສາິມ
ມາດຕິ
ຳ່ໍ ດຕຶດ
ດຕຫ໋ຕຫົາາຄ່
ຄໍົົນ
ນໄດ້
ໄດ້..
ែមេ គេន៍ះ ៎ យឆ៍ងែតមនឆ់ងដល់មនុ ស េឡ់ យ។៑ ី៑ ុ
β=.9: ໄຂ້៎ ຫວັດໃຫຍ່ແມ່ນ ພ່ະຍ່າດຕຶດຕຂື
ະໜາດສູູງ ແຕ່ບສາິມຳໍ ດຕຶດຕຫົາຄໍົນໄດ້.
Hindi, Myanmar, and Khmer had a relatively high amountclean:
สนามแข่
ง
ม้
า
แร็
น
ด์
ว
ค
ิ
ถู
ก
ปดและคาดว่
าจะยังคงปดอยูต่ ่อไปอีกถึง 2 เดือน
clean: ໄຂ້ຫວັດໃຫຍ່ແມ່ນ ພະຍາດຕິດຕຂະໜາດສູງ ແຕ່ບສາມາດຕິດຕຫາຄົນໄດ້.
of noise per sentence compared with Bengali, Lao, andclean: สนามแข่งม้าแร็นด์วคิ ถูกปดและคาดว่าจะยังคงปดอยูต่ ่อไปอีกถึง 2 เดือน
β=.9:

β=1:
β=1:

สนามแข่
แร็นด์วດคิ ຕຂະໜາດສູ
ถูกปดและคาดว่
าจะยังคงปดอยู
ต่ ่อไปอีกถึง 2 เดือน
ໄຂ້ຫວັດໃຫຍ່ແงມ່ม้ນาພະຍາດຕິງ ແຕ່ບສາມາດຕິ
ດຕຫາຄົນໄດ້.

Table
3 Perturbed
samples.
The
first
one
is cleanand the folThai. As a result, the performance of the NMT models on
β=1:สนามแข่แร็ด์วດคิ ຕຂະໍ
ถูกໜปดและคาดว่
ต่ ่อไปอีกถึง 2 เดือน
β=.95: ໄຂ້ຫาວັมแข่
ດໃຫຍ່ແงມ່ม้ນาພ່າດສູ່່ງ ແຕ່ບສາໍ
ມາດຕິ່ าจะยั
ດะຕຫ໋ຄ່ົນໄດ້.
β=.95:
สนํ์แະร็ຍ໋นາດຕິ
ปดและคํ
าดวาจ่
ยังາคงปดอยู

lowersare perturbed
with 𝛽ູງof
1, 0.95
andາຄໍ0.90
in top-down order.
β=.9: ໄຂ້ຫาວັมแข่
ດໃຫຍ່ແงມ່ม้
ະร็ຍ່น
າດຕຶແຕ່ບສາິມຳໍ ດຕຶ
ົນໄດ້.
Hindi, Myanmar, and Khmer was significantly impacted
β=.95:สนํด์วດคิี ຕຂືถูະ๋กໜາດສູ
ปดและคํ
าดวาจ่
ยังงคงปดอยํ
คงปดอยู่ ่อไปอี
β=.9:สนาํม่ນาພ່์แ
แร็ปดและค่
ดว์า่จດ๎ะຕຫົ
ูตไปอิกถึง 2 เดือน
Bengali:
β=.9:สนาํมแข่
ด์กวปดและคาดว่
คี ถู๋กปดและค่
า่จ๎ะยัต
ูต่อไปอิกถึง 2 เดือน
compared with Bengali, Lao, and Thai. The amount ofclean: สนามแข่
งม้งาม่
แร็านแร็
ด์วน
ค
ิ ถู
าจะยัางดว์
คงปดอยู่ งคงปดอยํ
่อไปอีกถึง 2 เดือนclean: สนามแข่এই ু งখুม้বาแร็ই সংিক এ মান
েষরদেহ ছড়ােত
ন อ।น
นด์วค
ิ ামক
ถูกปดและคาดว่
าจะยังคงปดอยู
ต่ ่อไปอีกপাের
ถึง 2 เดืnoise could be increased using highly similar glyphs withβ=1:clean:
এইามแข่িক এาดวาจ่
মানะু েষরদেহ ছড়ােতন อ।นু งখুม้ু বา์แইร็সং
β=1:
েยัষর
β=.95: สนํ
นด์วค
ิ ামক
ถูกปดและคํ
งคงปดอยู
ต่ ่อไปอีกপাের
ถึง 2 เดื𝛽 < 1, as demonstrated in Table 3. However, it is imporβ=.9:
β=1:

นด์วค
ี ামক
ถู๋กปดและค่
งคงปดอยํ
ูต่อঁ ไปอิน।এই มแข่ই সংিক এาดว์মানদেহ ছড়ােতন নঁอ।◌ু งখু
β=.95: สนาํখূม่ু বาแร็
মঁา่จান๎ะু ্েยัষর
ছড়ােতกপাের
পঁถึง◌2ােরเดื

tant to ensure that these perturbations are still readable byু ্েছড়ােত
β=.95:
এই ু ু খু বখূইুঁ বসংই সংামকএিক মানেষরএ মঁদেহ
া◌নঁ ানষরপঁ ◌পঁ
াের
β=.9:
সঁামক
ং িক
েষর দেহ
দেহছড়ােরনঁ ◌
নঁঁ ◌। ।ঁ ◌ােত
clean: এইপােরন ছড়।ঁ ােত
humans so that robustness studies are realistic.

Bengali#noises

20

Myanmar

β=.95:
β=1:
β=.9:
β=.95:
β=.9:

0

Khmer

Lao

Thai

10
0

clean:
लेे कननहঁ ◌ােতहो পঁसकता
β=1:
े मनु याদেহम ছড়
β=.9: এইलू ু बेे हখূदঁ বसंই সঁ ংामक
ামকहैैিকএ इसेমঁ ◌ঁানেষরাের নঁঁ ◌है।ै ।
β=1:
β=.95:
clean:

10
20#noisesHindiু েষর মঁদেহ
β=1:এই। ঁ ◌ােত পঁ াের নঁঁ ◌ ।
β=.9:এই ু ু খুু বখূইবসংই সঁামকং িকামকএিক মানুএ
◌ঁানছড়ােতেষর পাের
দেহন ছড়
Hindi: ইঁ সংामक
clean:ले कनया मছড়नह
हो◌াের
सकता
β=.95: এইलू ু बे हখূदু বसंামকहैিকএ इसेমঁ ান্ুেमनुষর দেহনঁ ◌ ।है ।ঁ ােত পঁ0.0

1.0 0.0β=.9:लू बेे हद सं ामक हैै लेे कन इसेे मनु या म नह हो सकता हैै ।हैै लेइसेे कनमेे नहैहैी ।हाॆ सकताेे हैै ।लूलूबेबेहेदहदसं सामकामकहै ले कनमनु इसेया ॆ ममऩेनह याहो सकता
हैहेैे लेइसेेॆ कनमेे नहै
लूलूबेेबेहेदहदसं सामकामकहैै लेे कनया ॆे ममऩेनह याहो सकताके मनुन इसेमऩनहॅहैीॉै ।हाॆहॉै सकताे
सकतॅ ीे हैै ।लूलूबेेबेहेदहदस सामकामकहैै लेे कनया े मेमऩे नहै ी याहाॆ सकताे
हेैे लेइसेॆ कॆ मऩेन इसेमेे नहॅे हैॉै ।हॉै सकतॅ ी हैै ।लू बेे हद स ामक हेैे लेॆ कन इसेे मऩ या मेे नहॅ ॉ हॉै सकतॅ ी हैै ।

clean: Myanmar:တုတ်ေကွး သည် အလွန ် အလွယ်တကူ ကူးစက် ေသာ်လည်း လူသားမ း သိ မထုတ်လင့် ငို ်ပါ ။

clean:
တုတ
တ်ေ်ေ
း သည်
နယ
် အလွ
ကူေသာ်
ကူးလ
စက်
ေသာ်
လည်
း လူ
ို ်ပါ ။
clean: တု
ကွကွ
း သည်
အလွအလွန ် အလွ
်တကူ ယကူ်တးစက်ည်း လူ
သားမ
း သိ
မထုသ
တားမ
်လင့် ငို း်ပသိါ ။ မထုတ်လင့် င
β=1:တုတ်ေကး သည် အလန် အလယ်တကူ ကူးစက် ေသာ််လည််း လူသားမ း သိိ မထုတ်လင့် ငို ်ပါ ။

β=1:
တု
ကး
သည်
အလန်
အလယ်
တကူ ကူတ
းစက်ည််း လူ
သားမ
း သိ
မထုသ
တားမ
်လင့် ငို း်ပသိါ ။ ိ မထုတ်လင့် င
β=1:
တုတ
တ်ေ်ေ
ကး
သည်
အလန်
အလယ်
ကူေသာ်
ကူး်လ
စက်
ေသာ်
်လည်
်း ိ လူို ်ပါ ။
β=.95: တံံုတ်ေကး သည်် အလန်် အလယ််တကံူ ကူႉစက်် ေသာ်ဲလည််း လသားမ ႏ သိ် မထံံုတ်လငွ ့် ငီှ ်ပါံ
β=.95: တံံုတ်ေကး သည်် အလန်် အလယ််တကံူ ကူႉစက်် ေသာ်ဲလည််း လသားမ ႏ သိ် မထံံုတ်လငွ ့် ငီှ ်ပါံ ။တံံုတ်ေကး သည်် အလန်် အလယ််တကံူ ကူႉစက်် ေသာ်ဲလည််း လသားမ ႏ သိ် မထံံုတ်လငွ ့် ငီှ ်ပါံ
1.0β=.95:
β=.9: တုၘတ်ေကၠႏ သည်် အလဲန ် အလယွ ်တကဲ ကူႈစက်် ေသာ်ဲလညိး လ်သူ ာံ းမျ် ား သိ် မထုၘတ်လင့ ငဳှ ်ပါဲ ။

1.0 0.0

β=.9:တုၘတ်ေကၠႏ သည်် အလဲန ် အလယွ ်တကဲ ကူႈစက်် ေသာ်ဲလညိး လ်သူ ာံ းမျ် ား သိ် မထုၘတ်လင့ ငဳှ ်ပါဲ ။တုၘတ်ေကၠႏ သည်် အလဲန ် အလယွ ်တကဲ ကူႈစက်် ေသာ်ဲလညိး လ်သူ ာံ းမျ် ား သိ် မထုၘတ်လင့ ငဳှ ်ပါဲ ။
Figure 5 Average amount of noise per sentence on the test set
5/17/23, 3:02 PM Khmer:
noisy_samples.html
with various 𝛼.
β=.9:clean: េមេ គេនះ5/17/23, 3:02 PM

Table 2

bg

hi

Base

16.9

SR

BLEU results on clean inputs.

my

km

lo

th

25.5

19.8

21.8

15.7

20.3

17.1

26.7

19.9

22.3

16.9

20.8

SR𝑐𝑡

19.5

28.9

22.0

24.2

18.8

22.6

IGE

18.7

28.2

21.5

24.0

19.1

22.5

DGE

19.0

28.0

21.4

24.4

18.7

22.4

β=1:
clean: េមេclean:
េមេ
β=.95:
េមេ់β=1:

5/17/23, 3:02 PMយឆងែតមន
។់ នុ ស េឡីយnoisy_samples.html
ិ ឆងដលម
noisy_samples.html
គេនះ យឆងែតមន
ឆងដល
ម
នុ
ស
េឡ
យ
់
ិ
ី ។
គេនះ ៑ យឆង
ែតម
ន
ឆងដល
ម
នុ
ស
េឡ
់
យឆង
ែត
ម
ឆួ
ងដល
េ
ឡ
ិ
ីឺ យ។
៍
់
់
៎

β=1:
់ នុ ស
β=.9:ែមគេន៍ះ ៎៑ យឆង
យឆ៍
ែតម
ឆងដល
ង
ិី ឆងដល
់ ដលម
β=.95: េមេេមេ់៎ គេនះយឆងឆួេ៍ ឡីឺ់ យ។់ ែតម់៑ ុ េឡ
៎៑ ន
ﬁle:///Users/hour/Documents/workspaces/adv_nmt/abugida/noisy_samples.html
1/1
β=.95:
េម
េ
គេនះ
យឆ
ង
ែត
ម
ន
ឆួ
ងដល
ម
នុ
ស
៑
៍ ឡឺ់ យ។
់
់
់
់
៎
ិ
β=.9: ែម ៎ គេន៍ះ ៎ យឆ៍់ ដល ៑ ុ េេឡ៑ ី ឆង
Lao:clean:
ໄຂ້
ຫ
ວັ
ດ
ໃຫຍ່
ແ
ມ່
ນ
ພະຍາດຕິ
ດຕຂະໜາດສູ
ງແຕ່ບສາມາດຕິ
ດ
ຕຫາຄົ
ນ
ໄດ້
.
β=.9: ែមេ៎ គេន៍ះ ៎ យឆ៍ងែតម់ ដលម់ នុ ស៑ នី ឆង៑ ុ េឡ់យ។
ﬁle:///Users/hour/Documents/workspaces/adv_nmt/abugida/noisy_samples.html
clean:
ໄຂ້ຫວັດໃຫຍ່ແມ່ນ ພະຍາດຕິດຕຂະໜາດສູງ ແຕ່ບສາມາດຕິດຕຫາຄົນໄດ້.
β=1:
ﬁle:///Users/hour/Documents/workspaces/adv_nmt/abugida/noisy_samples.html
clean:
ໄຂ້ຫວັ
ດໃຫຍ່ດຕຂະໜາດສູງ
ແຕ່ບສາມາດຕິດ
ນໄດ້
.
β=1:
ໃຫຍ່ແ
ແມ່
ມ່ນ
ນ ພະຍາດຕິ
ພະຍາດຕິ
ດຕຫາຄົ
ຕຫາຄົ
β=.95:
ໄຂ້ຫວັດ
ດໃຫຍ່ພ່ ະຍ໋າດຕິດຕຂະໜາດສູ
ຕຂະໍໜາດສູ
່່ງ ແຕ່ບສາມາດຕິ
ສາໍມາດຕິ
່ຕຫ໋ າຄ່ນົ ໄດ້.
β=1:ຫວັດໃຫຍ່ແມ່ນ ພະຍາດຕິງ ແຕ່ບສາມາດຕິ
ດຕຫາຄົ
ໄດ້.
β=.95: ໄຂ້ຍ໋າດຕຶ
ດຕິດດຕຂະໜາດສູ
ຕຂະໍະໜາດສູ
ໜາດສູ່ສາໍມຳາດຕິຕຫ໋ າຄໍ
ຄ່ນ
β=.9:ໄຂ້ຫວັດໃຫຍ່ແມ່ນ ພ່ ະຍ່ຕຂືູ່ງ ແຕ່ບສາິໍ່ ດຕຶດຕຫົ
ົນໄດ້.
β=.95:
ໄຂ້
ຫ
ວັ
ດໃຫຍ່
ແ
ມ່
ນ
ພ່
ະ
ຍ໋
າ
ດຕິ
ດ
ຕຂະໍ
ໜ
າດສູ
່
່
ງ
ແຕ່
ບ
ສາໍ
ມ
າດຕິ
່
ດ
ຕຫ໋
າ
ຄ່
ົ
ນ
β=.9: ໄຂ້ຫວັດໃຫຍ່ແມ່ນ ພ່ ະຍ່າດຕຶດຕຂືະໜາດສູູງ ແຕ່ບສາິມຳໍ ດຕຶດຕຫົາຄໍົນໄດ້
ໄດ້..
clean:
สนามแข่
ง
ม้
า
แร็
น
ด์
ว
ค
ิ
ถู
ก
ปดและคาดว่
าจะยังคงปดอยู
ต
่
่
อ
ไปอี
กถึ
β=.9: ໄຂ້ຫວັດໃຫຍ່ແມ່ນ ພ່ ະຍ່າດຕຶດຕຂືະໜາດສູູງ ແຕ່ບສາິມຳໍ ດຕຶດຕຫົາຄໍົນໄດ້
. ง 2 เดือน

Table 2 summarizes the performance of all NMT models
Thai:clean:
สนามแข่งม้าแร็นด์วคิ ถูกปดและคาดว่าจะยังคงปดอยูต่ ่อไปอีกถึง 2 เดือน
β=1:
clean:
สนามแข่
ง
ม้
า
แร็
น
ด์
ว
ค
ิ
ถู
ก
ปดและคาดว่
าจะยังคงปดอยู
ต่่ ่ ่ อ
on a clean test set. Our results show that SR outperformed
β=1:ิ ถูกปดและคาดว่
าจะยั
อไปอี
ไปอีก
กถึ
ถึงง 2
2 เดื
เดือ
อน
น
β=.95: สนามแข่
สนํามแข่งม้าแร็
์แร็นด์วค
ปดและคําดวาจ่
ะยังคงปดอยูต

β=1:
ิิี ถู
คงปดอยู่่ ่่ อthe baseline for all languages and consistency training furβ=.95:
สนํ
ม้
์แ
ร็น
นด์
ด์ว
วค
ค
ถู๋ก
กปดและคาดว่
ปดและคํ
คงปดอยูต
อไปอี
ไปอี
กถึ
ถึงง 2
2 เดื
เดือ
อน
น
β=.9: สนามแข่
สนาํามแข่งงม้
ม่าาแร็
แร็ปดและค่าดวาจ่
ดว์าาจะยั
่จ๎ะยังงงคงปดอยํ
ูตไปอิก
β=.95:ร็นิี ถูคงปดอยููต่ ่่ อ
β=.9: สนํสนาํามแข่
มแข่งงม้
ม่าา์แ
แร็
นด์
ด์ว
วค
ค
ถู๋ก
กปดและคํ
ปดและค่าาดวาจ่
ดว์า่จ๎ะะยั
ยังงคงปดอยํ
ต
อไปอี
ไปอิก
กถึ
ถึงง 2
2 เดื
เดือ
อน
น
ther boosted performance. Our training strategy with both

IGE and DGE was comparable with SR𝑐𝑡 .

clean:ই าসংএ মানেষর
পাের ูตন ่อ।ไปอิกถึง 2 เดือนু খুงবม่
β=.9: এইสนาํมแข่
แร็นামকด์วคี িকถู๋กปดและค่
าดว์าদেহ
่จ๎ะยัছড়ােত
งคงปดอยํ
clean:
এই ু খুু বই সং ামক িক এ মানু েষর
β=1:েষর দেহ ছড়ােত পাের ন ।clean:
β=1:
এই ুু খুখুবইই সংসং ামকামক িক
িক এ
এ মান
মান
েষর দেহদেহ ছড়ােত
ছড়ােত
পাের
ন নঁ।।◌ ।
β=.95: এইখূু বমঁ ানু ্েষরছড়ঁােত পােরপঁ ◌ােরনুু ্েেষর
β=1:
পাের
β=.95:
এই ুু খুখূুুঁ ববইই সং
সংামক িকিক এএ মানমঁ া◌নঁ ান
ষর
দেহ
ছড়
পঁ ◌পঁ
ােরঁ ােত
β=.9: এইসঁ ং ামকেষর দেহ
দেহছড়ােত
ছড়ানেরনঁ।◌নঁঁ ◌। ।ঁ ◌ােত
ুু্
β=.95:
এই
খূ
ব
ই
সং
ামক
িক
এ
মঁ
া
ন
ে
ষর
দেহ
ছড়
ােত
পঁ
◌
াের
নঁ
◌
ু
ঁ
β=.9: এই ু খূঁ বই সঁ ং ামক িক এ মঁ ◌ঁানেষর দেহ ছড়ঁ◌ােত পঁ াের নঁঁ ◌। ।
clean: लू बे हद सं ामक है ले कन इसे मनु या म नह हो सकता है ।
β=.9: এই ু খূঁ বই সঁ ং ামক িক এ মঁ ◌ঁানেষর দেহ ছড়ঁ◌ােত পঁ াের নঁঁ ◌ ।clean:
लू बेे हद सं ामक हैै लेे कन इसेे मनु या म नह हो सकता हैै ।
β=1:clean:
β=1:
लू बेबेे हहदद संसंामक हैहैै लेलेे कनकन इसे
इसेेॆ मनु
मनु
या म
म
नह
हो
सकता
β=.95: लूस ामकमऩे यामेे नहनहै ी होहाॆ सकता
सकताेे हैहैै ।।
β=1:
β=.95:
β=.9:
β=.95:β=.9:लूलू बेबेेे हहदद संसलूलू बेबेेे हहदद ससामकामक हैहैहेैैे लेलेेेॆ कनकनइसेेॆे मनुमऩेया ममेे नह
नहै
हाॆ
सकताे
कन इसेमऩ यानहॅ ीॉ होहॉै सकता
सकतॅ ीे हैहैैै ।।
ामक
है
ै
ले
े
कन
इसे
ॆ
मऩे
या
मे
े
नहै
ी
हाॆ
ामक हेैे लेॆ कन इसेे मऩ या मेे नहॅ ॉ हॉै सकताे
सकतॅ ीे हैहैैै ।।

β=.9:लू बेे हद स ामक हेैे लेॆ कन इसेे मऩ या मेे नहॅ ॉ हॉै सकतॅ ी हैै ।
clean: တုတ်ေကွး သည် အလွန ် အလွယ်တကူ ကူးစက် ေသာ်လည်း လူသားမ း သိ မထုတ်လင့် ငို ်ပါ ။

clean: တုတ်ေကွး သည် အလွန ် အလွယ်တကူ ကူးစက် ေသာ်လည်း လူသားမ း သိ မထုတ်လင့် ငို ်ပါ ။
β=1:ေကွကးး သည်သည် အလွ
အလန်
အလယ်
တကူ
ကူ ကူ
ကူးးစက်စက် ေသာ်
ေသာ်လ်လည်ည််းး လူလူသသားမားမ းး သိသိိ မထု
မထုတ
တ်လ
်လင့
င့်် င
င်ပါါ ။။
clean: တုတုတတ််ေန ် အလွ
ယ်တိုို ်ပ
β=1:တုတ်ေကး သည် အလန် အလယ်တကူ ကူးစက် ေသာ််လည််း လူသားမ း သိိ မထုတ်လင့် ငို ်ပါ ။work
is licensed
theူ ကူ
author(s)under
BY 4.0 ႏ သိ် မထံံုတ်လင
β=.95:
တ်ေ်ေကး
ကးThisသည်် အလန်
အလန်် အလယ်
အလယ်by်တကူကံစက်် ေသာ်
ေသာ်
ဲလည်
ည်CC
လသားမ
ွ ့်် ငင်ပါါံ ။―565 ―တံ
β=1:
တုံုတ
သည်
တ
ကူးႉစက်
်လ််းး လူ
သားမ း သိိ မထုတ်လင့
ိုီှ ်ပ
(https://creativecommons.org/licenses/by/4.0/).
β=.95: တံံုတ်ေကး သည်် အလန်် အလယ််တကံူ ကူႉစက်် ေသာ်ဲလည််း လသားမ ႏ သိ် မထံံုတ်လငွ ့် ငီှ ်ပါံ
β=.9:
န ်် အလယွ ််တသူ ာံ းမျ် ားႏ သိဳ ှ ီ ှ ်ပ
β=.95: တုတံၘံုတတ််ေေကၠကးႏ သည်
သည််် အလဲ
အလန်
အလယ်
တကဲ
ကံူ ကူ
ကူႈႉစက်
စက််် ေသာ်ေသာ်ဲဲလ
လညိည််းး လ်လသားမသိ်် မထု
မထံၘတ
ံုတ်လ
်လင့
င
ွ ့် င
င်ပါဲါံ ။