$\textit{SelectNoise:}$ Unsupervised Noise Injection to Enable Zero-Shot Machine Translation for Extremely Low-resource Languages
Authors: Maharaj Brahma
Kaushal Kumar Maurya
Maunendra Sankar Desarkar
Venue: EMNLP 2023 Findings (2023)
Keywords: Machine Translation
Low resource languages
Noise Injection
Zero-shot
BPE
TL;DR: We proposed novel noise injection method for extremely low resource languages to English machine translation task.

Abstract: In this work, we focus on the task of machine translation (MT) from extremely low-resource language (ELRLs) to English. The unavailability of parallel data, lack of representation from large multilingual pre-trained models, and limited monolingual data hinder the development of MT systems for ELRLs. However, many ELRLs often share lexical similarities with high-resource languages (HRLs) due to factors such as dialectical variations, geographical proximity, and language structure. We utilize this property to improve cross-lingual signals from closely related HRL to enable MT for ELRLs. Specifically, we propose a novel unsupervised approach, $\textit{SelectNoise}$, based on $\textit{selective candidate extraction}$ and $\textit{noise injection}$ to generate noisy HRLs training data. The noise injection acts as a regularizer, and the model trained with noisy data learns to handle lexical variations such as spelling, grammar, and vocabulary changes, leading to improved cross-lingual transfer to ELRLs. The selective candidates are extracted using BPE merge operations and edit operations, and noise injection is performed using greedy, top-p, and top-k sampling strategies. We evaluate the proposed model on 12 ELRLs from the FLORES-200 benchmark in a zero-shot setting across two language families. The proposed model outperformed all the strong baselines, demonstrating its efficacy. It has comparable performance with the supervised noise injection model. Our code and model are publicly available.

SELECTNOISE:
Unsupervised Noise Injection to Enable Zero-Shot Machine Translation for Extremely Low-resource Languages
Maharaj Brahma"
Kaushal Kumar Maurya Maunendra Sankar Desarkar
Natural Language and Information Processing Lab (NLIP)
Indian Institute of Technology Hyderabad Hyderabad, India
{cs23resche1004 c cs18resch1 1003 1003chht10 ac. in, maunendra@cse. iitiiitthh... ac. in
Abstract
ENG: Nadal's head to head record against the Canadian is 7-2, HIN: कनानडडपा d Rिeer सा सीयय रिकोड 7-2 š
this work, we focus on the task of machine translation (MT) from extremely low-resource language (ELRLs) to English. The unavail- ability of parailel data, lack of representation from large multilingual pre- pre-trained models, and limited monolingual data hinder the develop- of MT systems for ELRLs. However, many ELRLs often share lexical similarities with high-resource languages (HRLs) due to factors such dialectical variations, geograph- ical proximity, and language structure. We uti- lize this property to improve cross- coos-lingual sig- nals from closely related HRI. to enable MT for ELRLs. Specifically, we propose a novel unsupervised approach, SELECTNOISE, based on selective candidate extraction and noise in- jection to generate noisy HRLs training data. The noise injection acts regularizer, and the model trained with noisy data learns to handle lexical variations such as spelling, grammar, and vocabulary changes, leading improved cross- tross-lingual transfer to ELRLs. The selective candidates are extracted using BPE merge oper- ations and edit operations, and noise injection is performed using greedy, top-p, and top-k sampling strategies. We evaluate the proposed model on 12 ELRLS from the FLORES-200 -200 benchmark in a zero-shot setting across two language families. The proposed model outper- formed all the strong baselines, demonstrating its efficacy. has comparable performance with the supervised noise injection model. Our code and model are publicly available'.
N-HIN:
कनाे लिरागा नडा क सोया रिोड 7-2 हार
BHO:
पानाड के Faergy नाडार & रे रिा 7-2 àे पाा
Random Charader Injection (Lescca Simiarty 0.001
ENG:
Nadal's head to heed record against the Canadian 7-2,
HIN:
कनानििन के लिकापर सडोरर का रीया रिकंड 7-2 ô1
N-HIN:
कनमंडप के लि नाडार in सीोप ीिकंड 7-2 बा
BHO:
कनाड Raterge साडार के -.... रिां 7-2 रे बा SELECTNOISE Medel (Lodtcal Simlarty 0.70)
Figure I: Illustration character noise injection with random baseline (Aepli and Sennrich, 2022) and pro- pose SELECTNOISE model. The SELECTNOISE en- hances lexical similarity between noisy HRL (N-HIN) and ELRL (BHO). ENG: English, HIN: Hindi, N-HIN: Noisy Hindi and BHO: Bhojpuri languages. Red: Inser- tion, Blue: Deletion and Green; Substitution operations.
lingual corpora (Koehn and Knowles, 2017). How- ever, the linguistic diversity across the world vast, with over 7000 languages spoken". This linguistic landscape includes long tail of languages (Joshi et al.. 2020) that face significant challenges due to the lack of available resources for model develop- ment and are referred as extremely low-resource languages (ELRLs). ELRLs present unique chal- lenges for the development of MT systems as they lack parallel datasets, are excluded from large mul- tilingual pre-trained language models, and possess limited monolingual datasets. Consequently, the majority of research efforts in NMT have primarily focused on resource-rich languages (Bender, 2019), leaving ELRLs with limited attention and fewer vi- able solutions. Towards these concerns, this work is positioned as a step towards enabling MT tech- nology for ELRLs. Primarily focused on zero-shot setting for scalability.
1 Introduction
The modern neural machine translation (NMT; Aharoni et al. (2019); Garcia et al. (2021); Siddhant et al. (2022)) has achieved remarkable performance for many languages, but their performance heavily relies on the availability of large parallel or mono-
More recently, there has been active research to develop MT systems for LRLs. One direction is
"Equal contributions
code and model checkpoints link: https://github com/maharajbrahma haaerrrrrammeseeeetnntsseee
Phttps://www/ http://wwwwettroolou...../.o

---

multilingual training-based models (Aharoni et al., 2019; Garcia et al., 2021; Siddhant et al., 2022). These models are trained with multiple HRL lan- guages, enabling cross- looos-l transfer capabilities to improve translation performance for LRLs. An- other line of work focuses on data augmentation techniques (Sennrich et al., 6a; Wang et al., 2018) to generate more training data. However, these methods do not fully exploit the lexical simi- larity between HRLs and ELRLs. Many HRLs and ELRLs exhibit surface-level lexical similarities due to dialect variations, loan words, and geographical proximity (Khemchandani et al., 2021). For exam- ple. the word "Monday" is somvar in Hindi and somar in Bhojpuri. They are lexically very similar. To leverage this lexical similarity, recent studies have explored techniques like learning overlapping vocabulary (Patil al., 2022) or injecting random noise (Aepli and Sennrich, 2022; Blaschke al., 2023) in HRLs to resemble LRLs. These methods are only evaluated for natural language understand- ing tasks (NLU) and not for MT, which is a more challenging task. Inspired by these advancements, in this paper, we propose a novel unsupervised noise injection approach to develop an MT system for ELRLs.
should be comparable to the supervised noise in- jection model.
Our key contributions are: (1) We propose a novel unsupervised selective character noise in- jection approach, SELECTNOISE, to enable and improve MT for ELRLS to English. The injec- tion of selective candidate noise facilitates better cross cross-lingual transfer for ELRLS in the zero-shot setting. (2) We have developed an unsupervised mechanism to extract candidate characters based on BPE merge operations and edit operations. Fur- thermore, the noise injection employs greedy, top-k, and top-p sampling techniques, ensuring diversity. (3) We evaluated the model's performance using 12 ELRLS from the FLORES-22 evaluation set across two typologically diverse language families, Evaluations were conducted with both automated and human evaluation metrics. (4) The proposed model outperformed all baselines and has compa- rable performance with the supervised selective noise injection- based MT model. Additionally, we performed several analyses to demonstrate the ro- bustness of the proposed model.
Methodology
This section presents the details of the proposed SelecTNoISE model. As discussed in section I, the SELECTNOISE model has two stages; Selective Candidate Extraction and Noise Injection. In the selective candidate extraction stage, the noise in- jection candidate characters are extracted through an unsupervised approach. Specifically, we con- sider small monolingual data for HRL (D+) and for related (lexically similar) LRLs (D Dr. com- prises small monolingual datasets from multiple extremely ELRLS. During the process of building the Byte Pair Encoding (BPE) vocabulary, we ex- tract the BPE operations separately for the HRL (B+) and ELRLs (Bc). Next, we design an al- gorithm A extract selective candidate charae- ters from By and Bc and store them in candi- date pool Sc. Candidates are extracted with an approach inspired by edit-operations. In other words, we obtain Se as a result of A (Bx, Bz). In noise injection stage, the candidates are sam- pled from Se and injected into the source sentences of HRL corpus (H) from the large parallel data H. lang(e) En).
The proposed model is based on character noise injection and consists of two stages: Selective Can- didate Extraction and Noise Injection. In the selec- tive candidate extraction phase, candidate charac- ters are extracted in an unsupervised manner using small monolingual data from closely related HRI. and ELRLs. It relics on BPE merge operations and edit operations that take into account lexical similarity and linguistic properties. In the noise injection phase, noise is injected into the source side of parallel data of HRL using greedy, top-k, and top-p sampling algorithms. This noise injection serves as a regularizer and model trained with this noisy HRL data enhances robustness to spelling, grammar, or vocabulary changes and facilitates im- proved cross-lingual transfer for ELRLs. The eval- uation is done in the zero-shot setting, ensuring scalability. The proposed model is referred as the SELECTNOISE: Unsupervised Selective Noise In- jection model. Fig. I illustrates the effect of noise injection with SELECTNOISE model. In this paper, we investigate two hypotheses: (a) the selective noise injection model is expected to outperform random noise injection, and (b) the performance of the selective (unsupervised) noise injection model
Pr. {(h, la)n(())))) The injections are done using noise function n, resulting in noise- inbss--neettedd (augmented) parallel data: PH = {(h, e) (a)g(()) = M. lang(e) =

---

HRL
Clinlyy Rebaed
I.
Targel HRL E'
Zero-shot Generation Targel Brs
G-AL
Transforners Decoder
Transfomers Decoder
Gasoor vicropetir
Pool:
Transformers Encodar
M
Transformers Encoder
M
Candidaao Sampling: Groady, Top-k, Top-p
Source Noisy HRL
Leam Vocabulary
(a) Model Training
(b) Model Inference
Figure 2: Overview of the proposed SELECTNOISE model for extremely low-resource MT
En), where Hi = 17()). The H. acts as proxy training data for ELRLS. A BPE vocabulary V is learned with H. Then, we train the standard encoder-decoder transformers model (M; Vaswani et al. (2017b) from scratch with H. and V to obtain trained model M. Finally, zero-shot evaluation is done for ELRLs with M and Now, we present details of each model component. The overview of the proposed SELECTNOISE model is depicted in Figure 2.
deletions, and substitutions, and are referred to as the selective candidates.
As discussed, the extracted selective candidates are stored in the candidate pool Sc, dictionary data structure encompassing HRL and ELRL char- acters. The Sc consists of HRL characters, ELRL characters, edit operations, and their respective fre- quencies. An element of Se has following tem- plate: C : {I [I: fins, D fdet, S {ch f1,C2 f2, fk]]. The operations are: insertion (I), deletion (D) and substitution (S). The character G: represents the 1th element of Se, which is HRL character, ci ... dg denote the corresponding substituting candidates from ELRLs and f' is the associated frequencies. A few examples of selec- tive candidate extraction are illustrated in Fig. 3. Sample candidate pool (S.) is shown in Fig. 6. Intuitively, with this candidate pool Se, we have learned transformative entities that can be used to resemble an HRL to lexically similar ELRLs, which results bridging the lexical gap between HRL and LRLs. Training with such modified HRL data enhances the effectiveness of cross- lingual transfer signals for ELRLs. As candidates are extracted by considering the vocabulary word- formation strategy from BPE and edit operations, they indirectly consider the linguistic cues/informa- tion.
2.1 SELECTNOISE: Unsupervised Noise Injection
The formal procedure for unsupervised noise in- jection is presented in Algorithm 1. In the next subsections, will deep dive into each stage of the proposed model in details:
2.1.1 Selective Candidate Extraction
The first stage in the proposed approach involves extracting candidate characters that will subse- quently be utilized for noise injection. Given DH and De. we extract all BPE merge operations BH and Be, respectively. Each merge operation sists of tuples ((p.)) q)) By and ((r, s)) e BH- We pair each merge tuple of By with each tuple B& (i.e., cartesian setup). By and By have n and m merge operations, respectively, we ob- tain a total of / = m n pairs. We consider only those pairs where either and or q and s are the same while discarding the rest. For the considered tuples ((p,)) 9), (r, s)), calculate the character- level edit- edit-distance operations between non-similar elements of the tuple. For instance, if p and are the same, the edit operations are obtained using and elements. These operations are collected in the candidate pool S., which includes insertions,
2.1.2 Noise Injection to HRL
In the second stage, we sample selective candidates from S. and inject into the source sentences of HRL corpus (H) from the parallel dataset PH En} using a noise function y, resulting in a noise injected (aug- mented) parallel dataset PH = {(h, ()))agg(h)) = H. lang(e) = En), where = (?()). Details
{hhe (a)ggg())))
H. lang(e)

---

Hidd
ticularly those related to ELRLS. (ii The noise is added for small portion of characters (5-10%), making the HRLs training data closely resemble how sentences appear in ELRLs. As a result, the model is able to do a robust cross-lingual trans- fer to the ELRL in a zero-shot setting. In another perspective, the injection of noise acts as regular- izer (Aepli and Sennrich, 2022), contributing to overall enhancement in the model's 's performance.
will elaa, er (ii
de alrare All
Marge sperationn
(. pet T...
(a BI
1-0
Brogpad
2.2 Supervised Noise Injection
Figure 3: Illustration of selective candidates extraction for noise injection that utilizes BPE merge and edit operations. Here I. D. and S indicate insertion, deletion, and substitution respectively. Frequencies are associated with operations. O indicates the corresponding edit operation not extracted.
We have also investigated in supervised setting akin to the proposed SELECTNOISE approach. The key distinction lies in how the candidate pool Ssc is derived from limited parallel dataset between HRL and ELRLs. For each HRL and ELRL pair,
the noise function and candidate sampling strategy are presented below:
The noise injection function (1) Initially, we randomly se- lect 5%-10% of character indices from sentence E H. Subsequently, tween insertion, deletion, or substitution operations with equal probability. If the selected operation is insertion or substitution, we sample candidate character from S. to perform the noise injection op- eration. For deletion, the charter is simply deleted. These steps are repeated for all sentences in H to obtain the final H.
Noise Function:
Algorithm 1 SELECTNOISE: Unsupervised Noise Injection
is designed as follows:
uniformly choose be-
Require: [Inputs] HRL monolingual data Dy: closely re- lated ELRLs monolingual data Dz; number of merge opera- tions Mo: HRL parallel data (H. En); Noise injection candidate sampling strategy EETVVECAATC (AA
percentage range [p,, % 9:];
SM:
Ensure: [Output] Noisy source HRL
S. EXTTRACTSELLECCVECCADSS//
Dc.
for sentence & do
idzs 4- randomly select [p1% p2%) indices of for ida in idas do
ops t raanomml sample operation (insert, delete, substinate)
While noise in- jection for deletion operation, we simply delete the character. For insertion and substitution, we sample the candidate character for injection from S. using the greedy, top-p (mucleus), and lop-k sampling techniques inspired by decoding algorithms com- monly employed in NLG (Holtzman et al., 2019). Before applying these sampling techniques, the
Candidate Character Sampling:
if ops equals delete then Remove character at index ida
end
if ops equals Insert or ops equals substitute then
c sample candidate char, i.c., SM(SS, ops) Perform operation ops at index ida with e end if end for end for frequencies of the candidate characters are trans- formed into probability scores using the softmax operation. Intuitively, with the sampling technique, we aim to explore not only frequent candidate char- acters but also diverse candidates.
procedure EXTRAACTSELECTTEEAAASDDD Mo)
Initialize candidate pool S. <- to store candidates
Compute merge operations Bx BPE xx Mo)
Compute merge operations B; BPE(D Mo) for a in B. do for m in Bg de b where n= t ((p.g)) m tuple ((r,)))
n equals m or (p not equals r and not equals s) then
The performance of any learning model depends the quality of the training data. The presence of noise hampers the learning, and the outputs of the learned model exhibit the different nuances of the noise present in the data. In our specific case: (i) We train a model using data that contains noise, resulting in the model's increased robustness to minor lexical variations in different languages, par-
No operation is performed with n and m end p equals r then Compute edit-operations(q, 8) & update S. end if if equals then Compute edit-operations(p, r) & update S.
end if end for end for Return S. end procedure
aater conducting several ablation experiments, this range provides the best performance

---

Algorithm 2 Supervised Noise Injection
our case? (2) Does selective noise injection with the proposed SELECTNOISE model outperform the random noise injection model (Aepli and Sennrich, 2022)? (3) Does the model's performance persist across different language families? and (4) Does the unsupervised SELECTNOISE model demon- strate competitive performance with supervised ap- proach? Based these research questions, we have designed our experimental setup.
Require: [Inputs]
joint joint parallel data for all considered HRL- ELRL pairs (S E HRL parallel data Pa (H En); noise injection percentage range [pr % p2%] candidate sampling strategy SM
Ensure: [Output] Noisy source HRL H.
Sae JUUPEXTRRACSSLLECIIVEANDS
for sentence in H. do
idns +- randomly select [p: % -2%] indices for idæ: idas do
ops - -randomly sample operation (insers, delete, substitute)
if ops equals delete then
3.1 Datasets
Remove character index idæ
end if
The primary constraint of the proposed approach is to select closely related HRLs and ELRLs. With this criterion in mind, we have chosen two lan- guage families: Indo-Aryan and Romance. Within the Indo-Aryan family, we have selected Hindi (hi) as the HRL and ELRLs were Awadhi (awa), Bho- jpuri (bho), Chhattisgarhi (hne), Kashmiri (kas), Magahi (mag), Maithili (mai), Nepali (npi), and Sanskrit (san), based on their lexical similarity. For the Romance family, Spanish (es) served as the HRL, and the 4 ELRLS were Asturian (ast), Cata- lan (cat), Galician (glg), and Occitan (oci). We conducted separate experiments for each language family, training the model with the HRL to English MT task and evaluating in a zero-shot setting with corresponding ELRLs.
if ops equals Insert or ops equals substitute then
== sample candidate char, i.c., (ye(S,es ops) Perform operation ops at index idz with end if end for end for
procedure SUPEXTR SUVEEXTAACISSELERTVVAAAAYX
Initialize candidate pool S.c +- @ to store candidates for each ((s,e)) in Ps do Compute edit-operations(s, e) & update S.c. end for return Sec end procedure
we extract candidate set using edit operations and subsequently combine all the candidate sets in Sc. The rest of the modeling steps are similar to the SELECTNOISE. We hypothesize that the unsupervised method should exhibit competitive performance compared to the supervised approach. In the supervised candidate extraction, we assume the availability of limited amount of parallel data of approximately 1000 examples. A formal algo- rithm outlining in the Algorithm
In total, we have HRLS (English, Hindi, and Spanish) and 12 ELRLS. All the test datasets are sourced from FLORES-200 200 (NLLB Team, 2022), while the hi-en dataset is obtained from A14Bharat (Ramesh et al., 2022), and the es-en dataset is from Rapp (2021). The development set of FLORES- 200 was utilized as parallel dataset for supervised noise injection. A small amount of monolingual data was used for SELECINOISE and other base- line methods. Here, we used 1000 examples for each ELRL. Detailed dataset statistics and data sources are presented in Appendix D. In Appendix C. we provide an overview of the lexical similarity between HRLs and ELRLs.
2.3 Model Training and Zero-shot Evaluation
The stranded encoder-decoder transformers model (M) is trained from scratch using the noisy high- resource parallel dataset Py and V to obtain a trained model M. Where V is learned BPE vo- cabulary with Px. Subsequently, we use M to perform zero-shot generation for ELRLs. We have not used any parallel training data for ELRLs and directly employ M for inference, making this mod- eling setup zero-shot. The trained model transfers knowledge across languages, enabling coherent and meaningful translation for ELRLs.
3.2 Baselines
We compare the SELECTNOISE model with sev- eral strong baselines, including a traditional data augmentation model, lexical similarity based mod- els, and model based on random noise injection. Details of each baseline are presented below:
3 Experimental Setup
We designed our experimental setup to address the following set of questions: (1) Does noise injection improve performance for NLG tasks, i.e., MT in
Vanilla NMT: standard transformer-based
NMT model (Vaswani et al., 2017a) with BPE algorithm (Sennrich et al., 201

---

Models
bho
Indo-Aryan
kas
Romance
Average
Vanilla NMT
40.3
46.8
22.3
40.0
19.3
47.5
29.6
21.3
33.0
41.0
40.7
33.0
37.06
Wonl-drop
19.5
47.2
21.8
40.6
-9.0
47.6
18.6
20.6
37.6
43.6
43.4
36.0
37.
BPE-dhp
22.6
40.4
33.8
37.05
SwitchOut
20.1
29,0
ORPE
41.3
23,4
41.8
49.7
30.5
21.1
34.1
41.3
33.8
BPE-Dropout
39.8
47.4
22.5
39.9
49.6
47.7
29.3
21.2
33.2
40.8
41.4
33.00
37.15
Random Char Noise
40,9
48.4
23.8
40.8
50,0
47.5
31.2
21.9
40.9
46.1
46.4
38.2
39.68
SELECTNOISE Model
SELECTNOISE Groedy
25.2
49.9
42.0
SELECINOISE Top-k
42.4
26.0
41.5
47.8
41.06
SELECTNOISE Top
42.0
49.6
24.1
42.4
50.6
18.8
23.3
11.6
47.1
17.5
40.78
Supervised Noiser Injection
Selrctive mire Grandy
41.4
49.1
25.4
42.2
50.1
48.7
32.9
22.2
41.6
47.3
47.7
38.7
40.60
Selective Top-k
41.7
48.7
41.9
47.5
41.10
Selective nouse Top
41.4
27.3
48.9
41.6
48.2
41.35
Table 1: Zero-shot chrF (1) scores results for ELRLs English
Models
Indo-Aryan anag
Ronisace
Average
bhoo
hnc
san
mai
awa
npl
Kns
cat
glg
ast
oct
Vanilla NMT
11.1
17.2
2:7
10.1
18.5
18.3
5.1
2.6
5.3
10.1
12.3
9,86
Word-shop
8.7
13.7
1.9
7.7
15.2
16.1
3.01
1.6
6.9
10.7
13.3
6.5
8.76
BPE-drap
10.5
16.1
27
10.0
17.2
17.8
4.0
2.1
5.1
9.1
11.2
4.7
9,23
SwitchOst
4,3
7.7
1.4
4.9
8.4
7,9
2.9
1.2
3.5
6.3
5,04
2.9
RPE-Droqunl
11.6
17.5
3.1
19.3
18.3
5.4
2.5
10.1
Random Char Noise
12.
18.8
3.1
10.2
19.4
18.6
6.3
2.9
10.9
14.3
17.2
8.7
11.93
SELECINOISE Model
Sen ROTNNNSE Grealy
12.5
28.1
3.7
11.9
21.2
20.2
7.1
3.0
10.8
15.0
17.4
9.0
12,66
SELICTNOISE: Top-k
12.3
19.7
30.2
19.5
2.8
15.0
12:44
SELECTNOISE Top-p
19.5
3.2
14.8
Supervised Noise Injection Model
Selective noise Greedy
13.1
19.5
4.0
11,8
19.6
19.3
6.8
2.4
10.5
15.0
17.9
8:9
12.4
Selective moise Top-k
12.7
19.1
3.9
122
20.1
19.5
7.0
2.9
10.8
15.0
17.4
8.9
12.44
Selective noise Top-p
12.7
20.0
4.1
12.6
21.2
19.7
7.0
2.7
10.5
1544
18.1
12.76
Table 2: Zero-shot BLEU (1) scores results for ELRLs English
Models
Indo-Aryan
Romance
Average
bho
Inn
sam
mai
mag
21001
npi
kin
cal
asl
oci
0.295
Word-drop
0.197
0.533
0.357
0.198
0.551
0.563
0:417
0.353
0.361
0.140
0.654
0.312
0:545
RPF-dhop
0.506
0.533
0.367
0.509
0.557
0.572
0.422
0.363
0.316
0.415
0.452
0.285
0.440
SwithOut
0.411
I.446
0,318
0.415
0.467
11406
0,388
0.335
(1278
1.333
0,347
0.262
0,372
OBPE
0.502
0.525
0.571
0.502
0.561
0.436
0.306
0.404
0.416
0.266
0.438
BRE-Dropout
0.526
0.300
Ramdom Naris
0.547
0.501
0.569
0.441
380
0
0.491
0.319
0.467
SELECTNOISE Model
NOIEC Greedy
0.525
0.563
0.386
0.511
0,578
0.606
0.458
0.394
0.392
0.199
0,511
0.319
0.178
SELECTNOISE Top-k
0.584
0.558
0.386
0.507
0.576
11500
0:454
0.386
0.400
0.497
0.516
0.321
0:477
NEISEYNNOSSE: Top-p
0.505
(1388
0.508
Supervised Noise Injection Model
Selective muise Grealy
0.560
0,389
0.507
0.572
0,600
0:451
0.392
0.499
0.511
0.319
0:476
Selective maise Top-k
0.526
B..49
0,40
0.519
0.573
[1463
0,463
0.390
(1400
11494
0.306
0.326
00:467
Selective noise Top-p
0.524
0.558
0,400
0.510
0.584
0.455
0.455
0.386
0.391
0.501
0,512
0.321
0.466
Table 3: Zero-shot BLEURT (1) scores results for ELRLs English
Indo-Aryan
Average
Iine
mai
mag
31W33
npi
cal
ast
Nanilla NMT
11676
0.621
0.542
(1499
11533
0,497
0.411
drop
0,702
0.650
0.564
BIE--doo
0.683
0.687
0.197
0.615
0.711
0.732
0.551
0.400
0.138
0.515
0.505
0.389
0.560
SwishhOu
0.565
0.605
0.462
0.564
0.636
0.632
0.533
0.394
0-405
0.461
0.445
0.362
0.504
OSPE
0.664
1.676
0,452
0.630
0.707
0.740
0.544
0.392
01456
11534
18.01
0.400)
0.557
BPE: Dropout
0.644
0.672
0,471
0.615
0.710
0.537
0.505
0.534
0,500
0,411
Random Noise
0.611
0.559
StLEcrNoisE Modal
SELECYNOISE Greedy
0.672
0.714
0,893
0.647
0.735
0.765
0,575
0.412
0.525
0.020
0.598
0.434
0.509
NoISE Top-k
0,678
0.708
0.504
0.619
0.230
0.758
0.585
0.419
0.521
0.621
0.603
0.138
0.601
NoISE Top-p
0.677
0.559
0.902
0.643
0.730
0.758
0.586
0.411
0.326
0.635
0.600
0.442
0.568
Supervised Noise Injection Model
Selective noise Greody
0.522
0.603
Selective muise- Top-1
0.677
0.700
0.906
0.655
0,703
0.757
0.581
0.414
0.522
0,623
0.605
0.439
0.598
Selective muise Top-p
0.680
0.708
0.511
0,655
0.738
0.756
0.589
0.414
(1522
B.623
0.605
0.439
0.603
Table 4: Zero-shot COMET (1) scores results for EL.LSs - English

---

Models
Languages san
We further conducted the human evaluation to ensure the reliability of the performance gain. Three languages from the Indo-Aryan family (Bho- jpuri, Nepali, and Sanskrit) were selected based on their high, moderate, and low lexical similarity with the HRL (Hindi). To manage the annotators' work- load effectively, we limited our evaluation to three models: Vanilla NMT, BPE Dropout, and SELECT- NOISE. For each language, the human evaluation set consisted of 24 examples, and translations were obtained from above mentioned three models. Two annotators were employed for cach language to en- sure the inter-annotator agreement, and two ratings were obtained for each example from these annota- tors. All annotators held at least master's degree, were native speakers of the respective language and demonstrated proficiency in English. We use Crosslingual Semantic Text Similarity (XSTS) met- ric (Agirre al., 2012), which is widely adopted in the MT research for human evaluation. The XSTS metric employs a 1-5 evaluation scale, where rep- resents a very bad translation and represents a very good translation.
bho
opi
Annotator sct-1
Vanila NMT
3.54
2,42
2.21
BPE Dropout
3.29
2.37
1.83
SELECTNOISE Model
4.17
2.83
2,50
Anmotator sct-2
Vamilla NMT
3,42
1.96
2.17
BPE Dropout
2.79
1.96
SELECTNOISE Model
3.54
2.17
2.21
Table 5: Human evaluation results
Average score
Word-drop (Sennrich al., 2016a): In this baseline, 10% words embeddings from each sentence of the source iRL set to zero. This is a data augmentation technique to create training data ELRLs. The rest of the steps are similar to the SELECTNOISE model.
BPE-drop:
This approach is similar to the word-drop baseline but uses BPE tokens in- stead of words.
SwitchOut (Wang et al., 2018): this baseline, we randomly replace 10% of the source and target words with randomly sam- pled words from their respective vocabularies. We use the officially released implementation.
4 Results and Discussions
OBPE (Patil et al., 2022): OBPE modifies the learned BPE vocabulary incorporate overlapping tokens from both HRL and LRLs, even if the token is not frequent. We utilized the official implementation.
In this section, we will discuss results, observations and findings. The zero-shot automated evaluation scores are presented in Tables 1, 2, 3 and 4. The results are reported with greedy, top (k = 50), and top-p (p = 0.25) sampling strategies. Table 5 presents the human evaluation results.
BPE Dropout (Provilkov et al., 2020): It is based on the BPE algorithm to learn the vocabulary and generates non-deterministic segmentations for input text on-the-fly during training. We use a dropout value of 0.1.
DELECTNOISE vs. Baselines: The proposed and other models that incorporate lexical similar- ity have demonstrated superior performance com- pared to the Vanilla NMT model. While gen- eral data augmentation techniques like Word-drop and SwitchOut exhibit performance similar to the Vanilla NMT model, they perform poorly when compared to OBPE and BPE-Dropout models. These results indicate the importance of consider- ing monolingual data from ELRLs in the modeling However, random noise injection and the SELECT- NOISE approach outperform the OBPE and BPE- Dropout models, indicating the effectiveness of noise injection-based modeling techniques. In con- clusion, the careful selection of noise candidates, as done in the SELECTNOISE approach, has out- performed the random noise model (second best) and emerged as the state-of-the of-the---he-rr art model.
Random Char Noise (Aepli and Sennrich, 2022): This baseline methodology is similar to the proposed SElECTNOISE approach; but, noise injections are done randomly.
3.3 Evaluation Metrics
All the model performances are compared using both automated and human evaluation metrics. In line with recent research on MT for LRLs, we em- ploy two types of automated evaluation metrics (NLLB Team, 2022: Siddhant al., 2022). Specif- ically, lexical match-based metrics: BLEU (Pap- ineni et al., 2002) and chrF (Popovié, 2015) and learning-bas based metries: BLEURT (Sellam et al., 2020) and COMET (Pu et al., 2021).
Selective vs. Random Noise injection: Unsuper- vised selective noise injection approaches exhibit a

---

Performance vs. Sampling Strategies: The per- formance with different sampling techniques is compared, and it is observed that the greedy ap- proach for SELECTNOISL performs better for the majority of languages. This finding indicates the ex- larger performance gain compared to the random noise injection model. This observation empha- sizes the importance didate extraction and noise injection process.
Top-k sampling
Top-p sampling
systematic selective can-
Chrii 16
San
scores
30
san
mag
meg
kas
Lexical vs. Learned Evaluation Metrics: We observe strong correlation between lexical match- based metrics, such BLEU and chrF scores. Fur- ther, semantic-based metries like BLEURT and COMET exhibit similar trends lexical match metrics, indicating a high level correlation. This emphasizes the reliability of evaluation scores.
25
30
k-values
p-values
Figure 4: Proposed model performance trends with various k and values from top-k and top-p sampling respectively.
Language
Data size
BLEU
chri
997
19.5
49.6
hnc
Automated vs. Human Evaluation: The pro- posed SELECTNOISE model outperforms both baselines in human evaluation across all three lan- guages. The model demonstrates acceptable zero- shot performance for ELRLs, with strong correla- tion with automated evaluation scores.
6000
20.3
50.3
997
42.4
mai
6000
12.4
43.2
997
6.7
33.6
npi
6000
7.2
33.8
Table 6: Model performance with larger monolingual data
Performance across Language Families: Un- supervised selective noise injection consistently outperforms all the baselines across ELRLs, with few exceptions. The model exhibits similar perfor- mance trends across both language families. Unsupervised vs. Supervised Noise Injection: The unsupervised SELECTNOISE model performs comparably to the supervised model, with slight variations depending on the language and family.
Performance Trend with Top-k and Top-p: In Figure 4, the performance trend of the proposed model with varying values of k and p for top-p and top-k sampling depicted. The candidate pool consists of a maximum 61 characters (a range for k-value selection). The model performs best with a k-value of 50 and p-value of 0.25, offering valuable insights for optimizing its performance through parameter selection.
The performance gap between the two models is minimal, indicating their equal strength.
Impact of Monolingual data size: The proposed SELECT model relics on the small monolin- gual dataset of ELRLs. We investigate the impact of large monolingual dataset the model's per- formance for ELRLs. Table 6 demonstrates that larger dataset leads performance boost, sug- gesting the extraction of more meaningful noise injection candidates.
istence of one-to-one lexical mapping across HRL and ELRLs. However, other sampling approaches are also effective for subset of ELRLs.
Language similarity Vs. Performance: Figure illustrates the comparative trend of lexical sim- ilarity score between ELRLs and HRLs and per- formance (ChrF score). It can be observed that lexically similar languages boost the model's per- formance, leading to an improved cross-lingual transfer for the SELECTNOISE model. For exam- ple, languages like Kashmiri (kas), which have the lowest similarity, exhibit the lowest performance, whereas Chhattisgarhi, with the highest lexical sim- ilarity, demonstrates the highest performance.
Overall Performance: As we can observe from the average automated evaluation scores, the pro- posed SELEETNNISEE model outperforms all the baselines by significant margin. It also exhibits comparable performance to the supervised model, and this performance persists across different lan- guage families. These findings satisfy our hypoth- esis, leading us to conclude that the proposed SE- LecTNoisE model is a state-ofther art model for English-to-ELRRLs MT systems.
Performance with Less related Languages: We evaluate the zero-shot translation performance of Vanilla NMT and proposed SELECTNOISE models with two relatively less lexically similar ELRLs.
5 Further Analyses
In this section, we perform detailed analysis with SElEcTNoISE to understand factors contributing to performance gain and analyze robustness.
These two languages belong to distinct language families, namely Bodo (Sino- Tibetan) and Tamil

---

ceptable performance for many LRLs. However, adapting these models to ELRLs is challenging be- cause they often lack parallel data, have limited monolingual data, and are absent from mPLMs. contrast, we propose a model that only requires a small monolingual data (1000 examples).
Similaiity ChiF
Siniarity Scores (in peromttio)))
50
30
Data Augmentation for Low-resource MT: The limited availability of parallel data leads to a wide range of data augmentation approaches (Zhang et al., 2019; Gao et al., 2019; Cotterell and Kreutzer, 2018). Traditional approaches include perturbation at the word level, such as word dropout (Sennrich al., 2016a), word replacement (Wang et al., 201 8) and soft-dccoupling (SDE; Wang al. (2019)) to improve the cross- tross-lingual transfer for LRLs. Such perturbation acts as regularizer and enhances robustness to spelling variations; how- ever, their impact is limited (Aepli and Sennrich, 2022). In a different research direction, noise injection-based modeling (Sperber et al., 2017; Karpukhin et al., 2019) has been explored to test the robustness of MT systems. More recently, lex- ical match-based models have been explored improve the cross-lingual transfer by vocabulary overlapping (Patil al., 2022), non-deterministic segmentation (Provilkov et al., 2020) and noise in- jection (Acpli and Sennrich, 2022; Blaschke et al., 2023). In contrast to these methods, we propose linguistically inspired systematic noise injection approach for ELRLS
20
0
glg
Extremely low resource languages
Figure 5: Language similarity Performance.
Language
Model
BLEU
chrF
Vanilla NMT
2.4
18.2
Bodo
SELECTNOISE
18.7
Vanilla NMT
0.6
11.7
Tamil
SELECTNOISE
0.9
13.3
Table Zero-shot translation performance of Vanilla NMT vs. SELECTNOISE on less related LRLs with HRL (Hindi)
(Dravidian). Bodo has Devanagari script, while Tamil employs script conversion to match HRL (Hindi) script. The results are reported Table 7. It is observed that the performance gain is min- imal due to the dissimilarity of ELRLs with the corresponding HRL.
6 Related Work
MT for Low-resource Languages: Limited paral- lel corpora for many LRLs lead to active research in multilingual MT. These are grounded with transfer learning to enable cross-lingual transfer (Nguyen and Chiang, 2017; Zoph et al., 2016) and allow related languages to learn from each other (Fan et al., 2021; Costa-jussà et al., 2022: Siddhant et al., 2022). Further, these approaches can be extended by grouping training data based relat- edness (Neubig and Hu, 2018) clustering simi- lar languages (Tan et al., 2019) improve perfor- mance for LRLs. In another direction, monolingual corpora are combined with parallel сorpora to en- hance translation quality (Currey et al., 2017) or used for unsupervised NMT (Artetxe et al., 2018; Lewis et al., 2020), reducing the reliance on par- allel data. Back-translated data is also widely uti- lized for training MT systems for LRLs (Sugiyama and Yoshinaga, 2019; Edunov al., 2018). More recently, models powered by large multilingual pre- trained language models (mLLMs) enable MT with limited language resources (NLLB Team, 2022; Zhu et al., 2023). These models have shown ac-
Conclusion
This study presents an effective unsupervised ap- proach, SELECTNOISE, for cross-lingual transfer from HRLS to closely related ELRLs through sys- tematic character noise injection. The approach involves extracting selective noise injection candi- dates using BPE merge operations and edit opera- tions. Furthermore, different sampling techniques are explored during the noise injection ensure diverse candidate sampling. The model required only a small (1K example) monolingual data ELRLs. The proposed model consistently outper- formed strong baselines across 12 ELRLS from two diverse language families in the ELRLs-to-Enggl MT task. The cumulative gain is 11.3% (chrF) over vanilla NMT. Furthermore, the model demon- strated comparative performance noise injection model. In the future, we will extend SELECTNOISE to English-to- to-ELL MT task, as well as other NLG tasks and languages.
supervised

---

Limitations
A14Bharat, Jay Gala, Pranjal A. Chitale, Raghavan AK. Sumanth Doddapaneni, Varun Gumma, Aswanth Ku- mar, Janki Nawale, Anupama Sujatha, Ratish Pudup- pully, Vivek Raghavan, Pratyush Kumar, Mitesh M. Khapra, Raj Dabre, and Anoop Kunchukuttan. 2023. Indictrans2: Towards high-quality and accessible ma- chine translation models for all 22 scheduled indian languages. arXiv preprint Xivi 2305. 16307.
The present study is based on the assumption of closely related high-resource languages and ex- tremely low-resource languages to facilitate im- proved cross-lingual transfer. However, the pro- posed method may not be effective for ELRLs that have a different script and lack a script conversion or transliteration tool. Additionally, the model's s performance may be suboptimal for languages that share the same script but exhibit significant lexical differences, such Hindi and Bodo. Furthermore, this study focuses on ELRLS English translation, and it remains to be explored whether the noise injection approach is beneficial for the English to ELRLs translation task.
Mikel Artetxe, Gorka Labaka, Eneko Agine, and Kyunghyun Cho. 2018. Unsupervised neural ma- chine translation. In Proceedings of the Sixth Inter- national Conference on Learning Representations.
Emily Bender. 2019. The# benderrule: On naming the languages we study and why it matters. The Grudient, 14.
Verena Blaschke, Hinrich Schütze, and Barbara Plank. 2023. Does manipulating tokenization aid cross- lingual tramsfer? a study on POS tagging for non- standardized languages. In Tenth Workshop on NLP for Similar Languages, Varieties and Dialects (Var- Dial 2023), pages 40-54, Dubrovnik, Croatia. Asso- ciation for Computational Linguisties.
Acknowledgements
We extend our heartfelt gratitude to all human eval- uators whose meticulous assessment of the transla- tion performance of the proposed model and vari- ous baselines played a pivotal role in the success of this research. Furthermore, we wish to express our deep appreciation to the anonymous reviewers and the esteemed meta-reviewer for their invaluable and insightful suggestions, which greatly improve the quality of the manuscript.
Marta R Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling machine translation. arXiv preprint arXiv:2207.066) 04767..
Ryan Cotterell and Julia Kreutzer: 2018. Explaining and generalizing back traksatann through wake-sleep. arXiv preprint 180.04200
References
Anna Currey, Antonio Valerio Miceli Barone, and Ken- neth Heafield. 2017. Copied monolingual data im- proves low-resource neural machine translation. In Proceedings of the Second Conference on Machine Translation, pages 148-156, Copenhagen, Denmark. Association for Computational Linguistics.
Noëmi Aepli and Rico Sennrich. 2022. Improving zero- shot cross-lingual transfer between closely related languages by injecting character-level noise. In Find- ings of the Association for Computational Linguis- tics: ACL 2022, pages 4074 4774, Dublin, Ireland. Association for Computational Linguistics.
Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding back-translation at scale. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 489-500, Brussels, Belgium. Association for Computational Linguistics.
Eneko Agiree, Daniel Cer, Mona Diab, and Aitor Lenzalez-Agirre, 2012. SemEval-2012 task 6: A pilot semantic textual similarity. In #SEM 2012: The First Joint Conference on Lexical and Compu- tational Semantics - Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantie Evaluation (SemEval 2012), pages 385- 393, Montréal, Canada. Association for Computa- tional Linguistics.
Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, et al. 2021. Beyond english-centric mul- tilingual machine translation. J. Mach. Learn. Res.. 22(107):
Roee Aharoni, Melvin Johnson, and Orhan Firat. 2019. Massively multilingual neural machine translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 3874 3884,3884, Minneapolis, Minnesota. Association for Computa- tional Linguistics.
Fei Gao, Jinhua Zhu. Lijun Wu, Yingee Xia, Tao Qin, Xueqi Cheng, Wengang Zhou, and Tie-Yan Liu. 2019. Soft contextual data augmentation for neural machine translation. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics. pages 5539-5544.

---

Xavier Garcia, Aditya Siddhant, Orhan Firat, and Ankur Parikh. 2021. Harnessing multiinguulityy in unsu- pervised machine translation for rare languages. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Compula- tional Linguistics: Human Language Technologies, pages I 133, Online. Association for Computa- tional Linguistics.
Graham Neubig and Junjie Hu. 2018. Rapid adapta- tion of neural machine translation to new languages. In Proceedings of the 2018 Conference on Empiri- cal Methods Natural Language Processing, pages 875-880, Brussels, Belgium. Association for Com- putational Linguistics.
Toan Q. Nguyen and David Chiang. 2017. Trans- fer learning across low-resource, related languages for neural machine translation. In Proceedings of the Eighth International Joint Conference on Natu- ral Language Processing ( (Volume Short Papers), pages 296-301, Taipei. Taiwan. Asian Federation of Natural Language Processing.
Ari Holtzman, Jan Buys, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degener- ation. CoRR, abs/1904.00951..
Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury, 2020. The state and fate of linguistic diversity and inclusion in the NLP world. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282-6293, Online. Association for Computational Linguistics.
James Cross Onur Çelebi Maha Elbayad Kenneth Heafield Kevin Heffernan Elahe Kalbassi Janice Lam Daniel Licht Jean Maillard Anna Sun Skyler Wang Guillaume Wenzek Al Youngblood Bapi Akula Loic Barrault Gabriel Mejia Gonzalez Prangthip Hansanti John Hoffman Semarley Jarrett Kaushik Ram Sadagopan Dirk Rowe Shannon Spruit Chau Tran Pierre Andrews Necip Fazil Ayan Shruti Bhos- ale Sergey Edunov Angela Fan Cynthia Gao Vedanuj Goswami Francisco Guzmán Philipp Koehn Alexan- dre Mourachko Christophe Ropers Saftyyah Saleem Holger Schwenk Jeff Wang NLLBB Team, Marta R. Costa-jussà. 2022. No language left behind: Scaling human-centered machine translation.
Vladimir Karpukhin, Omer Levy, Jacob Eisenstein, and Marjan Ghazvininejad. 2019. Training on synthetic noise improves robustness to natural noise in ma- chine translation. In Proceedings of the 5th Work- shop Noisy User-generated Text (W-NUT 2019), pages 42-47, Hong Kong, China. Association for Computational Linguistics.
Yash Khemchandani, Sarvesh Mehtani, Vaidehi Patil, Abhijeet Awasthi, Partha Talukdar, and Sunita Sarawagi. 2021. Exploiting language relatedness for low web-resource language model adaptation: An Indic languages study. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1th International Joint Confer- ence on Natural Language Processing (Volume 1: Long Papers). pages 1312-1323, Online. Association for Computational Linguistics.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic evalu- ation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Compu- tational Linguistics, pages 311-318, Philadelphia. Pennsylvania, USA. Association for Computational Linguistics.
Vaidehi Patil, Partha Talukdar, and Sunita Sarawagi. 2022. Overlap-based vocabulary generation im- proves cross-lingual transfer among related lan- guages. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers), pages 219-233, Dublin, Ireland. Association for Computational Linguistics.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd Inter- national Conference on Learning Representations, ICIR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings.
Philipp Koehn and Rehecca Knowles. 2017. Six chal- lenges for neural machine translation. In Proceedings of the First Workshop on Neural Machine Translation, pages 28-39, Vancouver. Association for Computa- tional Linguistics.
Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedingy of the Tenth Workshop on Statistical Machine Translation, pages 392 395, Lisbon, Portugal. Association for Computational Linguistics.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. pre-training
Ivan Provilkov, Dmitrii Emelianenko, and Elena Voita. 2020. BPE-dropout: Simple and effective subword regularization. In Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guistics, pages 1882-1892, Online. Association for Computational Linguistics.
BART: Denoising sequence-to-sequee for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 7871-7880, Online, Association for Computa- tional Linguistics.
Amy Pu, Hyung Won Chung, Ankur P Parikh, Sehastian Gehrmann. and Thibault Sellam. 2021. Learning compact metries for mt. In Proceedings of EMNLP.
I. Dan Melamed. 1995. Automatic evaluation and uni- form filter cascades for inducing n-best translation lexicons. In Third Workshop on Very Large Corpora.
Gowtham Ramesh, Sumanth Doddapaneni, Aravinth Bheemaraj. Mayank Jobanputra, Raghavan AK,

---

Ajitesh Sharma, Sujit Sahoo, Harshita Diddee, Ma- halakshmi J. Divyanshu Kakwani, Navneet Kumar, Aswin Pradeep, Srihari Nagaraj, Kumar Deepak, Vivek Raghavan, Anoop Kunchukuttan, Pratyush Ku- mar, and Mitesh Shantadevi Khapra. 2022. Samanan- tar: The largest publicly available parallel corpora collection for 11 Indic languages. Transactions of the Association for Computational Linguistics, 10:145- 162.
Ashish Vaswami, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, L ukasz Kaiser, and Illia Polosukhin. 2017a. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
Ashish Vaswani. Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017h. Attention is all you need. Advances in neural information processing systems, 30.
Reinhard Rapp. 2021. Similar language translation for Catalan. Portuguese and Spanish using Marian NMT. In Proceedings of the Sixth Conference on Machine Translation, pages 292-298, Online.
Xinyi Wang, Hieu Pham, Philip Arthur, and Graham Neubig. 2019. Multilingual neural machine transla- tion with soft decoupled encoding. In International Conference on Learning Representations.
Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020. BLEURT: Learning robust metrics for text genera- tion. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7881-7892, Online. Association for Computational Linguistics.
Xinyi Wang, Hieu Pham, Zihang Dai, and Graham Neu- big. 2018 SwitchOut: an efficient data augmentation algorithm for neural machine translation. In Proceed- ings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 856-861 Brussels, Belgium. Association for Computational Linguistics.
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016a. Edinburgh neural machine translation sys- tems for WMT 16. In Proceedings of the First Con- ference Muchine Translation: Volume 2, Shared Task Papers, pages 371-376, Berlin, Germany. Asso- ciation for Computational Linguistics.
Zhirui Zhang, Shuangzhi Wu, Shujie Liu, Mu Li, Ming Zhou, and Tong Xu. 2019. Regularizing neural ma- chine translation by target-bidirectional agreement. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 443-450.
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016b. Neural machine translation of rare words with subword units. In Proceedings of the 54th An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), pages 1715- 1725, Berlin, Germany, Association for Computa- tional Linguistics.
Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Lingpeng Kong, Jiajun Chen, Lei Li, and Shujian Huang. 2023. Multilingual machine translation with large language models: Empirical results and analy- sis. arXiv prepriat arXiv:2304. 046655.
Aditya Siddhant, Ankur Bapna, Orhan Firat, Yuan Cao, Mia Xu Chen, Isaac Caswell, and Xavier Garcia. 2022. Towards the next 1000 languages in multilin- gual machine translation: Exploring the synergy be- tween supervised and self-supervised learning. arXiv preprint 2201022001.031 10.
Barret Zoph. Deniz Yuret, Jonathan May, and Kevin Knight. 2016. Transfer learning for low-resource neu- ral machine translation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Lan- guage Processing, pages 366--175 Austin, Texas. Association for Computational Linguisties.
Matthias Sperber, Jan Nichues, and Alex Waibel. 2017, Toward robust neural machine translation for noisy input sequences. In Proceedings of the 14th Interna- rional Conference on Spoken Language Translation, pages 96-96.
A Character Candidate Pool (S,)
Amane Sugiyama and Naoki Yoshinaga. 2019. Data augmentation using back-translation for context- aware neural machine translation. In Proceedings of the Fourth Workshop on Discourse in Machine Trans- lation (DiscoMT 2019), pages 35 35,44, Hong Kong, China. Association for Computational Linguistics.
A sample structure of character candidate pool S. is illustrated in Fig. 6. More details are presented in Section 2.1.1.
B Performance for HRLs
Xu Tan, Jiale Chen, Di He, Yingce Xia, Tao Qin, and Tie- Yan Liu. 2019. Multilingual neural machine translation with language clustering, In Proceed- ings of the 2019 Conference Empirical Methods Natural Language Processing and the 9th Inter- national Join Conference on Natural Language Pro- cessing (EMNLP-IJCNLP), pages 963-973, Hong Kong, China. Association for Computational Lin- guistics.
Table 8 analyzes the performance of the proposed model for HRLs across both language families. It demonstrates comparable performance with the vanilla NMT model for HRLS while boosting the performance of ELRLS. This highlights the effec- tiveness of the proposed model in handling both HRLs and ELRLs.

---

Languages chrF
Models
BLEU
BLEURT
COMET
Hindi (hi)
Vanilla NMT
33.4
60.2
0.724
0.868
Random Char Noise
33.0
59.5
0.722
0.865
SELECTNOISE
34.2
60.5
0.726
0.869
Spanish (es)
Vanilla NMT
21.5
53.5
0.695
0.810
Random Char Noise
21.7
53.0
0.689
0.806
SELECTNOISE
21.3
53.1
0.689
0.869
Table 8: Comparative performance for HRLs across both Indo-Aryan and Romance families.
0.45
0.58
0.59
0.32
0.58
0.53
0.38
0.32
Candidate Pool (S.) Templete
C:: f. D: fa. S: Es fa.
Es: fa 1b.
bho
hne
kas
mag
mai
npi
Cx: fa fa 8: Ez Ez fT...
(a)
Cptl:: fa. D: fa. S:: Br fz. Ez f1
0.37
0.25
0.45
0.32
C. C. S: Br
/4
E3: fa
cat
glg
ast
oci
Few Sample Elements of Candidate Pool (8.)
(b)
"etet 1D' :1561, (181 1482, .. (1 192, 191199, (tof) (1422
Figure 7: Lexical similarity heatmap between HRL and its related ELRLs. Fig. (a) depicts a similarity score for the Indo-Aryan family where HRL is Hindi. Fig. (b) depicts a similarity score for the Romance family where HRL is Spanish. Note: Darker color denotes more similarity.
for (2 1Dre1s, 181
S' (MP's 1D1s 33, 1811 (F)
Figure 6: Top a template for the character candidate pool Sc. The operations are: insertion (I), deletion (D) and substitution (S). The character 6; represents the 8th element of S.. which is an HRL character, e...ce dk denote the corresponding substituting candidates from ELRLs and is the associated frequencies. The Bottom shows a few sample elements the S..
OBPE baseline, we use dev set of FLORES-200 consisting of 997 as monolingual corpus for learn- ing the overlap vocabulary.
E Implementation Details
C HRL-ELRL lexical similarity measurement
Our vanilla NMT model is based on standard trans- former architecture consisting of 6 encoder and decoder layers. We trained our model for a maxi- mum epoch of 15. We use Adam (Kingma and Ba, 2015) optimizer with 31 = 0.9 and 83 = 0.98. We set a lcarning rate of 0.0005. We use a dropout of 0.2. We performed data normalization and prepro- cessing using IndicNLP library', We perform our experiments using fairseq library. For evaluation we use the lexical match-based BLEU metric" (Pa-
Figure 7 shows the lexical similarity between HRL and related ELRLs for both language families. Lex- ical similarity is obtained using the longest com- mon subsequence (LCS; Melamed (1995)).
D Datasets
Detailed statistics of datasets used in our experi- ments are shown in Table 9. For performing anal- ysis on less-related language, we use the general test set IndicTrans' (AI4Bharat et al., 2023) for Bodo and 20ORES200 test set for Tamil. For
tht htt.://ggtthuu...c wttp:ttt.b.oooooooouuuoaoa httph//owwommmooooooocctttttaaaa nlp library
'ht /hwbb//iitttbb.b com/pyyoorch/faiirsg 'nrefs: haw.a//mwxelleffooooooo/aooooooe

---

Languages
#Dev
#Test
HRL
Source
bho, mag, mai, npi, awa, san, kas, hne
997
1012
hi
FLORES-200
ast, cat, glg, oci
997
1012
es
FLORES-200
HR Lang Pair
#Train
Dev
#Test
Source
hi-en
10.1M
997
1012
A14Bharat
es-en
6.6M
997
1012
Rapp (2021)
Table 9; Dataset statistics and language details
pineni al., 2002), chrl? (Popović, 2015) metric. semantic- BLEURT (Sellam et al., 2020), and COMET (Pu et al., 2021) metrics.
F Sample Translations
Fig. 8 presents sample translations from Random Character Noise, SELECTNOISE and Supervised Character Noise injection models.
btteeeecwwcdliiiieeeeeeee/eeeeee "Reported using BLEURT20 checkpoint
"Reported using wm122-comet-da model

---

BHO: दम क पोसाा, पमी & शापपीी रिं सहप यडोअा से उमकर पहमन साकीीत के बाद अाड
ENG: The announcement made after Trump had phone conversation with Turkish President Recep Tayyip Erdogan.
Trump opened up about phone call Turkish President Recep Tayyip Erdogan.
UCN: Afler phone call with Turkish President Recep Tayyip Erdogan, Trump announced his decision.
SCN: Trump's announcement came after phone call with Turkish President Recep Tayylp Erdogan.
HNE: yी @ कतार करोद रिन से पद gu बमा
ENG: Police said body appeared
day.
RCN: The police said the body had been lying on the ground for the past few days.
UCN: police
lying on
past days.
SCN: police said body
lying past days.
ONU tamén ten previsto crear un fondo para axudar aos paises afectados polo quentamento global afrontar seu impacto.
ENG: The UN also intends set fund relieve the countries affected the global fear facing having serious impact.
RCN: The UN also planning to set up fund to tax years of countries affected by global pole to cope with or have an impact.
The UN also intends set up a fund to relieve the countries affected by the global fear of facing or having serious impact.
SCN: The UN also plans set up fund to relieve the countries affected by the global warming face their impact.
AST: Munchos cudadanos
manifestantes
desorde.
ENG: Several Bishkek residents blamed protesters from the south for the lawlessness.
RCN: Cubans Bishkek accused southern protestors of the deorde.
UCN: Many people in Bishkek accused the Swiss demonstrators disorder.
SCN: Many people Bishkok accused southem demonstrators of disorder.
Figure 8: Sample translations from various models ELRLS English MT direction. RCN: Random Character Noise injection model, UCN: Unsupervised Character Noise injection model (i.., SELECTNOISE model) and SCN: Supervised Character Noise injection model.


Reviews:
[{"review_id": "HhWqBWZLNV", "reviewer": "Reviewer_4tcT", "summary": "", "questions": "1. Have the authors consider other languages like Tibetan? I'd love to see more ELRLs languages.\n2. As the available metrics might not be trained on those involved ELRLs, I think you can provide more QE/metric results (e.g., COMETKIWI, UniTE) to support the significance of the improvements on model performance. You can refer to the official report of WMT Metrics/QE papers to find some methods.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the constructive feedback, and phrases like ’Well-written and easy-to-follow’' are encouraging. In the final version of the paper, we will address missing citations, grammatical and spelling errors, and all presentation issues.\n\n**On the \"less number of ELRLs\":**  We have used 12 ELRLs from 2 typologically diverse families (Indo-Aryan and Romance) to evaluate the performance of the proposed model (refer to section 3.1). Additionally, 2 LRLs (Bodo and Tamil) have been employed for model analysis from the Sino-Tibetan and Dravidian language families, resulting in a total count of 14 ELRLs across 4 language families. **The decision regarding the number of ELRL languages was influenced by recent literature and the availability of languages in the FLORES-200 test set. For example, Aepli, N., & Sennrich, R. (2021) employed 5 LRLs; Patil et al. (2022) used 12; Li, Zhaocong, et al. (2022) worked with 5; Khemchandani, Yash, et al. (2021) used 3; Kumar, Sachin, et al. (2021) worked with 7; Lin, Zehui, et al. (2021) used 4 languages, among others.** The proposed model consistently outperforms the best baseline for most of the languages, with an absolute average difference of 1.6 in chrF scores, providing evidence for the model's performance (see Tables 1, 2, 3, and 4 for detailed results). Additionally, in Table 5, we present human evaluations that further support and align with the findings from automated evaluation scores (which correlate with each other). Taking these into consideration, we believe that the chosen number of ELRLs is reasonable. The evaluation involving both automated and human scores adds credibility and lends trustworthy evidence of the model's improvement. In the future, we will explore with Tibetan and other ELRLs. \n\n**On the point of \"metrics might be insufficient\":** We have considered four widely used automatic evaluation metrics to assess various model performances. Specifically, we employ both metrics based on lexical overlap (chrF and BLEU) and metrics focused on semantics (BLEURT and COMET). The task addressed in this study involves translating from ELRLs to English, where the evaluation scores are reported by comparing the generated English text with Ground Truth English text across all evaluation metrics. The COMET metric also utilizes the source sentence (ELRLs). While COMET might not have been trained with ELRLs, it certainly includes related HRLs, resulting in reliable scores. Moreover, these metrics have generally demonstrated reliable performance in evaluating English sentences, as indicated in the previous literature. Additionally, we have conducted human evaluations to further enhance the reliability of the proposed model. Therefore, evaluating English text using metrics based on lexical overlap, metrics focusing on semantics, and human evaluation collectively contribute to the credibility of the proposed experimental and evaluation framework. Nonetheless, we have computed evaluation scores for both COMETKIWI and UniTE. Due to limitations in readability, we have included COMETKIWI scores here. UniTE exhibits similar trends and correlates with the other metrics.\n\nCOMETKIWI scores for Indo-Aryan Family\n| Models | bho | hne | san | mai |mag |awa | npi | kas |\n| -------- | ----- | ---- | ----- | ---- | --- | --- | --- | --- |\n| Vanilla NMT | 0.656 | 0.675 | 0.503 | 0.623 | 0.695 | 0.709 | 0.584 | 0.533 | \n| Random Char Noise | 0.667 | 0.684 | 0.514 | 0.633 | 0.700 | 0.705 | 0.595 | 0.528 | \n| Unsupervised Noise Injection | 0.670 | 0.688 | 0.517 | 0.630 | 0.701 | 0.717 | 0.602 | 0.537 | KIWI\n| Supervised Noise Injection | 0.669 | 0.689 | 0.530 | 0.641 | 0.707 | 0.719 | 0.606 | 0.530 | \n\nCOMETKIWI scores for Romance Family\n| Models | cat | glg | ast | oci |\n| -------- | ----- | ---- | ----- | ---- | \n| Vanilla NMT | 0.443 | 0.494 | 0.480 | 0.353 | \n| Random Char Noise | 0.495 | 0.568 | 0.562 | 0.403 | \n| Unsupervised Noise Injection | 0.499 | 0.584 | 0.581 | 0.407 | \n| Supervised Noise Injection | 0.495 | 0.586 | 0.584 | 0.403 |", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Proposed method relieve the reliance on available training resources.\n2. Well-written and easy-to-follow.", "reasons_to_reject": "1. Involved ELRLs might be less to derive convincing conclusion. More languages (especially different families or character-systems) are welcomed to help improve the manuscript. (See Q1)\n2. The used metrics might be insufficient as they might have little (or even contradictory) support on the performance measurements (See Q2)", "questions_to_authors": "1. Have the authors consider other languages like Tibetan? I'd love to see more ELRLs languages.\n2. As the available metrics might not be trained on those involved ELRLs, I think you can provide more QE/metric results (e.g., COMETKIWI, UniTE) to support the significance of the improvements on model performance. You can refer to the official report of WMT Metrics/QE papers to find some methods.", "paper_topic_and_main_contributions": "This paper investigates the possibility of building MT system from extremely low-resource languages to English. \n\n\n----After the rebuttal----\n\nThanks for the feedback from the authors!\nBy referring to the newly updated QE results, I believe that some of my concerns are well dispelled, and the effectiveness of the proposed method are strengthed. The authors can add those results into the revision for better claims.\n\nI'll raise my judgement on the soundness of this paper from 3 to 4.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A", "Typos_Grammar_Style_and_Presentation_Improvements": "N/A"}, "extra_text": {}}, {"review_id": "DDDwkMIHkm", "reviewer": "Reviewer_eEMd", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the constructive feedback. In the final version of the paper, we will address missing citations, grammatical and spelling errors, and all presentation issues.\n\n**On the point of \"limited novelty\":** To the best of our knowledge this is the first effort towards systematic unsupervised noise augmentation to enable zero-shot machine translation for ELRLs.  The novel contributing aspects through this work have not been seen in previous work: **(a) we proposed a novel approach based on BPE merge, edit operations and sampling techniques to find noise injection candidates which required only small (1000) monolingual examples for ELRLs. This modeling is automated, linguistically intuitive and scalable.  (b) We show that noise augmentation helps cross-lingual transfer in the NLG task like MT in extremely data-constrained scenarios. Previous work shows benefits for NLU tasks only (Aepli& Sennrich, 2022). (c) Our proposed  noise injection not only shows improvements in transfer among dialects but also between less-related languages like SAN and KAS (see Table 2). This opens up possibilities for further work in noise augmentation for cross-lingual transfer (for instance one-many translation scenarios, other NLG tasks, MT between related languages).** Different data augmentation methods have different motivations, and we systematically investigate transfer between related languages in low-resource settings as our motivation. Existing data augmentation techniques for the MT task mentioned in Section 2 have one or more of the following limitations: (1) may not aim for the zero-shot cross-lingual transfer, (2) may not aim for the MT task and (3) may require (small) parallel in ELRLs, (4) do not utilize lexical similarity. The proposed model focuses on the ELRL -> En MT task in a zero-shot setting and does not require any parallel for ELRLs. Nevertheless, we have considered multiple data augmentation-based state-of-the-art models as baseline models and compared the performance in Table 1-4. The proposed model outperformed all the baselines.\n\n**On the \"inclusion of a few ELRLs\":** Similar concerns have been raised by Reviewer #1. We kindly request you to refer to the response to Reviewer #1. We are unsure about what the reviewer is referring to with the phrase “the parameters of ratio in this method.” Assuming it pertains to the range of noise injection percentage, we would like to clarify that we have conducted a comprehensive set of ablation studies using different noise ranges. As a result, we have concluded that a noise injection percentage of 5-10% works best.\n\n**On the \"comparison with the back-translation or translations models\":** The proposed unsupervised selective noise injection does not require any parallel data and only 1000 monolingual data in ELRLs to enable zero-shot translation for ELRLs to En. For ELRLs, neither back-translated nor Translationese-based MT models can be built, as they require an MT system in ELRLs which is primarily the goal of this work: to enable an MT system for ELRLs. The assumption of an existing MT system in ELRLs can not be made. \n\n**On the \"definition of Zero-shot and few-shot\":** The proposed model only focuses on the zero-shot setting, not in a few-shot setting. Here, the Zero-shot translation (MT) is defined as the training of an MT model without any human-annotated parallel data. If the training of an MT system requires a few human-annotated parallel examples (typically 100 - 1000), it is referred to as a few-shot translation. In the proposed approach, we assume the absence of a human-annotated parallel dataset for ELRLs. Furthermore, by introducing noise into the HRLs human-annotated parallel, the modified parallel dataset acts as a pseudo-parallel dataset for multiple ELRLs, which is used to train the MT model from ELRLs to English. This pseudo-parallel dataset does not necessitate human annotation. As a result, the model we propose is referred to as a zero-shot translation model for ELRLs.\n\n**On the point of \"multi-sense words\":** Any research is incremental, so we aim to develop an MT system for ELRLs with this work. We assume that many instances of multi-sense words are present in the dataset, and a model trained with this dataset will leverage the broader context and learning to differentiate such words and provide appropriate translations. However, we have not focused on this or not conducted any analysis. This task can be taken separately as future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The authors try to address the translation between extremely low-resource languages (ELRLs) and English, and they apply char-level candidates' selective and noise injection strategies to conduct pseudo-parallel sentence pairs for ELRLs.\nThis method performs better than word-substitute methods.", "reasons_to_reject": "Although this method performs better than other word-substitute methods, some issues exist as follows:\n1. Few languages or language families are involved in experiments, does the method have the generalization on other low-resource languages or language families, and the parameters of ratio in this method are also suited?\n2. What are the advantages of this method, compared to the back-translation or translationese method on monolingual data?\n3. How to define zero-shot translation and few-shot translation.\n4. How to address multi-sense words in different contexts?", "questions_to_authors": "", "paper_topic_and_main_contributions": "In this paper, the authors aim to address the translation between extremely low-resource languages (ELRLs) and English. And the authors propose an unsupervised method by utilizing the shared char-level vocabulary to conduct pseudo-parallel sentence pairs for ELRLs.\nHowever, I don't think this simple data-augmentation method is novel in 2023, and the authors only evaluate the translation performance between ELRLs and English, which is inadequate to be the zero-shot translation.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "6Vxx0IPaEW", "reviewer": "Reviewer_2AWf", "summary": "", "questions": "A. Line 221-223: I imagine some HRL characters are associated more with some particular edit operation and less with others. Did you test biasing edit operation sampling depending on the HRL character?\n\nB. Line 267-269: Did you compute the softmax with raw counts? For example, if the candidate characters are \"a\" and \"b\", and they were observed 5 and 2 times respectively, their selection probabilities are exp(5) / (exp(5) + exp(2)) and exp(2) / (exp(5) + exp(2))? if so, frequent characters are more likely to be sampled. (exp(5) / exp(2)  ~ 20, 5 / 2 = 2.5)\n\nC. Line 386: \"Four languages ... namely Bhojpuri, Nepali and Sanskrit ...\" What's the forth language?\n\nD. Line 389-394: \"Each language's evaluation set comprised 24 examples, which were individually assessed by a single annotator. ... each evaluation set was evaluated by two annotators.\" I'm confused. How many human ratings were annotated to each example?\n", "limitations": "", "rating": 2, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the constructive feedback, and phrases like ‘method is linguistically intuitive’ are encouraging. In the final version of the paper, we will address missing citations, grammatical and spelling errors, and all presentation issues.\n\n**On the point of \"paper didn't provide detailed analyses\":**  The primary hypothesis we are operating is to enable MT technology for lexically similar ELRLs. In fact, the proposed model boosts the performance for less similar languages like San and Kas (from Table 1-4), and Bodo and Tamil (Table 7) languages. **On generalization capabilities, we have experimented with 12 ELRLs (Table 1-4) and compared them with different baseline models with four evaluation metrics, human evaluation (Table 5), experiments with different sampling algorithms, supervised experiments, performance for HRLs (Table 8 in the appendix), analysis with top-p and top-k values (Figure 2), analysis with different-sized monolingual data (Table 6), and performance analysis for less related languages (Table 7).** All these experiments and analyses provide evidence that the model generalizes across two language families and 12 ELRLs/dialects. Moreover, the proposed novel noise injection model based on BPE merge operations, edit distance, and sampling techniques is an automated process.  This can be applied to any ELRLs and language families as long as they have a small dataset (1000) of monolingual examples. With these points, we conclude that the proposed model's generalization capabilities can be easily scaled to any closely related ELRLs and HRLs.\n\n**On the point of \"The paper may not be appealing to a wide audience\":** The proposed model aims to advance and scale language coverage for MT research and present a robust foundational model for researchers for ELRLs. Although the proposed model requires similar scripts, there are many ELRLs that have related HRLs in the same script. Additionally, script conversion stands as a plausible alternative. With this work, our broader goal is to enable language technology for a general audience. As per Ruder (2022), there are around 7,000 languages spoken across the globe, and 400 languages have more than 1 million speakers, with approximately 1,200 languages having more than 100,000 speakers. Despite this, only around 100 languages are incorporated into large pre-trained models, and a few hundred have MT technology enabled. Furthermore, in ACL 2008, it was found that 63% of all papers focused on English. A recent similar study from ACL 2021 concluded that almost 70% of papers were only evaluated using English. Even a decade later, there has been little change. Considering these motivations, our paper is positioned as a step towards enabling MT technology for larger languages with a significant number of speakers (listed in the table below) that may interest the broader audience. Here are the number of speakers for considered languages (source: wikipedia):\n\n| Languages | # of speakers |\n| ---------------------------- | --------------- | \n| Bhojpuri (bho) | 50M |\n| Chhattisgarhi (hne) | 16M |\n| Magahi (mag) | 12M | \n| Maithili (mai) | 13M | \n| Awadhi (awa) | 3.8M |\n| Nepali (npi) | 2.9M | \n| Kashmiri (kas) | 7.1M | \n| Sanskrit (san) | 24, 821 | \n| Bodo (brx) | 1.4M | \n| Tamil (tam) | 69M |\n| Catalan (cat) | 9.2M | \n| Asturian (ast) | 5.5M | \n| Galician (glg) | 2.4M |\n| Occitan (oct) | 100,000 - 800,000 |\n\n**Response to  Question A:** We agree with the reviewer's observation on varying HRL-ELRL character associations. In our current setup, we've extracted edit operations with frequencies for multiple ELRLs. For a given HRL character “h” undergoing three operations, each with “m” character mappings across “n” ELRLs, we have 3xmxn options with different frequencies. Considering any explicit association (heuristics/rules) in MT training may encourage biases for some ELRLs. Moreover, this limits the proposed model's scalability to other language families. To tackle the imbalanced data operations associate, we introduce divergence by considering edit operations with equal probabilities and using sampling decoding techniques to add diversity. While our framework indirectly considers associations, automated explicit modeling of such associations is left as future work.\n\n**Response to  Question B:** Yes, the softmax computation employs the raw counts. In order to foster diversity and mitigate the tendency to sample the most frequent characters, we incorporate both top-k and top-p strategies in addition to the greedy approach.\n\n**Response to  Question C:**  It is a typo; we used three languages for human evaluation: Bhojpuri, Nepali, and Sanskrit.\n\n**Response to  Question D:** We apologize for the confusing language. For human evaluation, we have considered three languages: Bhojpuri, Nepali, and Sanskrit. For each language, we randomly sampled 24 examples for human evaluation. For each example, we obtained two ratings from two different annotators to ensure inter-annotator agreement. We have reported the average scores of each annotator in Table 5.\n\n**On the point of \"Model is Zero-shot\":** Similar question is raised by reviewer #2. We kindly request you to refer our response to reviewer #2.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* The proposed method doesn't require parallel corpora thus can be applicable to many low resource language pairs.\n\n* The proposed method is linguistically intuitive and has a potential to incorporate more linguistic knowledge.", "reasons_to_reject": "* The proposed method is expected to work better with LRLs whose linguistic structures are close to the base HRL, but less with other LRLs. Though the authors conducted experiments with many LRLs, the paper didn't provide detailed analyses. It is difficult to speculate how these findings will generalize into other language families.\n\n* As the authors acknowledged in Limitations, the proposed method is applicable only when the LRL has a HRL which is close not only in linguistic features but also scripts. The paper may not be appealing to a wide audience.", "questions_to_authors": "A. Line 221-223: I imagine some HRL characters are associated more with some particular edit operation and less with others. Did you test biasing edit operation sampling depending on the HRL character?\n\nB. Line 267-269: Did you compute the softmax with raw counts? For example, if the candidate characters are \"a\" and \"b\", and they were observed 5 and 2 times respectively, their selection probabilities are exp(5) / (exp(5) + exp(2)) and exp(2) / (exp(5) + exp(2))? if so, frequent characters are more likely to be sampled. (exp(5) / exp(2)  ~ 20, 5 / 2 = 2.5)\n\nC. Line 386: \"Four languages ... namely Bhojpuri, Nepali and Sanskrit ...\" What's the forth language?\n\nD. Line 389-394: \"Each language's evaluation set comprised 24 examples, which were individually assessed by a single annotator. ... each evaluation set was evaluated by two annotators.\" I'm confused. How many human ratings were annotated to each example?\n", "paper_topic_and_main_contributions": "The paper proposes a method to synthesize a pseudo parallel corpus for a low resource language (LRL) pair from a parallel corpus of a linguistically-close high resource language (HRL) pair. The method first build a database of HRL character edit operations which are likely to produce LRL (sub)words from HRL words in an unsupervised fashion. Then characters at random positions in the HRL parallel corpus are changed by a character edit operation selected from the database. \n\nThe experimental results show translation quality improvements consistently across serval languages against various vocabulary regularization techniques.", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "* Since the comparison to supervised noise injection is a part of the paper's main claim, Algorithm 2 should be included in the main part.\n\n* Some people might be confused by calling  the experiment setup zero-shot. My understanding is that this paper proposes a method to synthesize a pseudo ELRL-English parallel corpus from a HRL-English parallel corpus. With this framing, the experiment is not zero-shot.\n"}}]

Author rebuttal:

Meta-review: Reasons Accept:
The reviewers agree that this a simple and linguistically motivated data augmentation approach that could be straightforwardly applied to extremely low-resource language pairs.
Reasons Reject:
The proposed approach heavily relies on the existence of a high resource language which share linguistic features which limits its applicability. There could have been some analysis of how relatedness affects viability of the approach. 
MetaReview
The authors submitted extensive rebuttal including new results which led to the increase in soundness from one reviewer from 3 to 4. All reviewers acknowledged rebuttal and there was some final discussion.
Decision: Accept-Findings