Context Based Machine Translation With Recurrent Neural Network For English-Amharic Translation 
Authors: Yeabsira Asefa Ashengo
Rosa Tsegaye Aga
Surafel Lemma Abebe
Venue:  (2020)
Keywords: Context based machine translation
machine translation
Neural network machine translation
English to Amharic machine translation

Abstract: The current approaches for machine translation usually require large set of parallel corpus in order to achieve fluency like in the case of neural machine translation (NMT), statistical machine translation (SMT) and example-based machine translation (EBMT). The context awareness of phrase-based machine translation (PBMT) approaches is also questionable. This research develops a system that translates English text to Amharic text using a combination of context based machine translation (CBMT) and a recurrent neural network machine translation (RNNMT). We built a bilingual dictionary for the CBMT system to use along with a large target corpus. The RNNMT model has then been provided with the output of the CBMT and a parallel corpus for training. Our combinational approach on English-Amharic language pair yields a performance improvement over the simple neural machine translation (NMT).

Under review as a conference paper at ICLR 2020
CONTEXT BASED MACHINE TRANSLATION WITH RECURRENT NEURAL NETWORK FOR ENGLISH- AMHARIC TRANSLATION
Anonymous authors Paper under double-bling review
ABSTRACT
The current approaches for machine translation usually require large set of par- allel corpus in order to achieve fluency like in the case of neural machine trans- lation (NMT), statistical machine translation (SMT) and example-based machine translation (EBMT). The context awareness of phrase-based machine translation (PBMT) approaches is also questionable. This research develops a system that translates English text to Amharic text using combination of context based ma- chine translation (CBMT) and a recurrent neural network machine translation (RNNMT). We built bilingual dictionary for the CBMT system a large target corpus. The RNNMT model has then been provided with the out- put of the CBMT and parallel corpus for training. Our combinational approach on English-Amharic language pair yields performance improvement over the simple neural machine translation (NMT).
along with
INTRODUCTION
Context based machine translation (CBMT) is phrase-based machine translation (PBMT) approach proposed by Miller et al.] (2006). Unlike most PBMT approaches that rely on statistical occurrence of the phrases, CBMT works on the contextual occurrence the phrases. CBMT uses bilingual dictionary its main translator and produces phrases to be flooded into large target corpus.
The CBMT approach addresses the problem of parallel corpus scarcity between language pairs. The parallel corpus set for English-Amhart language pair, for instance, composes of the Bible, the Ethiopian constitution and international documents. These sources use words specific to their do- main and overlook phrases and words used by novels, news and similar literary documents. The CBMT uses synonyms of words place of rare words and rely on large target corpus and a bilin- gual dictionary to help with data MaarcityMMilae corpus like most PBMT such as the statistical machine translation (SMT)(rrown et al. 1990) and the example-based machine translation EBMT Gangaadhaaaa 2011). The CBMT, however, fails in fluently translating texts compared to the neural machine translation (NMT).
al. (006). It is not dependent on large parallel
The NMT learns the pattern of humans' translation using human translated parallel corpus. Its trans- lations are more fluent and accurate than all the rest so far when evaluated individually (Popovic) 2017.. However, NMT struggles to translate properly rare words and words not commonly used Wu et al. 2016). In addition, NMT requires large parallel corpus for training.
The aim of this research to build system by combining the CBMT with the NMT for English to Amharic translation. The combination of PBMT and NMT is the future and most promising than the individual approaches themselves (Popovic 2017). CBMT's ability to address rare words and the NMT's ability to produce fluent translation along with their context awareness makes them complementary couple.
The combination is done by providing the NMT with two inputs, one from the source language and the other from the output of the CBMT to produces the final target sentence. In this paper, we show that this approach utilizes the strength of each method to achieve significant translation perfor- mance improvement over simple NMT. The improvement is mostly dependent on the performance of the CBMT and mostly on the bilingual dictionary of the CBMT.

---

Under review as a conference paper at ICLR 2020
2
RELATED WORKS
PBMT approaches are mostly used to translate English to Amharic as in the case of Gasser 2012 Tadesse & Mekuria 2000), Teshome (2000), Besacier et al. (0000, Zewgnehi (2017) and Taye et al. (215)_ Below we summarize the researches with most significance to ours.
The SMT approach takes parallel corpus an input and it selects the most frequent target phrase based on statistical analysis for each searched source phrase (Brown et al., 1990). The SMT ap- proach applied the English-Amharic pair has produced 18.74 % BLEU score (Tadesse & Mekuria, 2000). The SMT has good accuracy in translating all the words in sentence but it not fluent (Ola- dosu al. 2016).
Hybrid of SMT and rule based machine translation (RBMT) translates and orders the source text based on the grammar rules of the target language and sends it to the SMT for final transla- tion(Yuliant) Yulianti et al.,
et al., (014). The hybrid approach for English-Amharic pair has achieved 15% improvement over SMT on simple sentence and 20% improvement for complex sentence ZZeggnch 2017). Hybrid of RBMT and SMT gets fluency from RBMT and accuracy from SMT but for longer sentences, the reordering failsoOladosu et al. 2016).
The CBMT approach has been implemented for the language pair Opanish-English In CBMT, the source phrases are translated using bilingual dictionary and flooded to target corpus. It has achieved 64.62% BLEU score for the researchers' dataset[Millee al., (2066) The CBMT outperforms SMT in accuracy and fluency but translation of phrases with words not in the bilingual dictionary is weak (Miller al. 2006).
The NMT has been researched by different groups and here the research by Googles' researchers on the language pair English-French is presented. The NMT model is trained using parallel corpus. The source sentence is encoded as a vector and then decoded with the help of an attention model. Googles' NMT model has achieved 38.95% BLEU score( et al., 2016). The NMT has accuracy and fluency but it fails to translate the whole sentence and also fails to perform well with rare words Wu et al., 2016). To solve this using sub-word units has been suggested (Sennrich et al. 2016) but Amharic has unique treats like "Tebko-lalto", one word with two different meanings, which can only be addressed by using context.
The NMT has been modified to translate low resourced languages. One approach uses universal lexical representation (ULR) were a word is represented using universal word embeddings. This benefits low resource languages which have semantic similarity with high resourced languages (Gu cct al.) 2018). This achieved 5% BLEU score improvement over normal NMT. However, most south- Semitic languages like Amharic do not have strong semantic relative with large resource. NMT has also been modified to work with monolingual corpus instead of parallel corpus using cross-lingual word AabedddiggArteteer et al., 2017). Such an approach achieved 15.56% BLEU score which was less that the semi-supervised and supervised which achieved 21.81% % BLEU score and 20.48% BLEU score respectively.
Combination of NMT and PBMT which takes the output of SMT (a PBMT) and the source sentence to train the NMT model has been used for the language pair English-German. It has achieved BLEU points over basic NMT and PBMT et al., 2016). Combination of NMT and PBMT which takes three inputs; output of basic NMT, output of SMT and output of Hierarchical PBMT (HPBMT) of SMT has been implemented for English-Ching points over basic NMT and 5.3 BLEU points over HPBMT (Zhang al. (017)). The combination of PBMT and NMT performs better (Popovic) 2017) in terms of accuracy and fluency but it is dependent on the performance of the chosen PBMT approach.
language pair. It achieved 6 BLEU
3 METHODOLOGGY
In this research, we have selected CBMT and NMT to form combinational system. This approach addresses the limitation of context unawareness of some PBMT approaches like SMT and the need for large parallel corpus of simple NMT. In our approach, the source sentence in English and the translation output of the CBMT in Amharic has been fed to the NMT's encoder-decoder model as shown in Figure T The NMT model then produces the final Amharic translation.
2

---

Under review as a conference paper at ICLR 2020
Source text
CBMI
target
targettext
Figure 1: The proposed combination method
The combination of the CBMT and the NMT follows the mixed approach proposed by Niehues al. (2016). Their mixed approach feeds the NMT with the source sentence and the output of the PBMT. The research by Zhang et al. (2017) also supports this way of combining different systems.
3.1
CBMT SYSTEM
The CBMT outperforms RBMT, SMT and EBMT when it comes to languages with less parallel corpora(Miller al. 2006). It uses a bilingual dictionary, large target corpus and smaller source corpus, which is optional. In the context based machine translation, there are different components working together produce the translation. Figure shows the flow of data in the different compo- nents of the CBMT.
N-gram generator of source sentence
Phrase Tran slator using Bilingual dictionary and synonym finder
Source
N-gram generator of Target text
Flooder
N-gram connector
CBMT target text
Figure 2: Overview of the CBMT system
The source sentence is converted into N-gram phrases and then it is translated using bilingual dic- tionary. CBMT's performance is mostly dependent the efficiency of the dictionary. We have manually built a phrase based dictionary aided by Google translate. A synonym finder helps the dictionary's search using WordNet(SSergel] 1998). WordNet library with large lexical database of English words. It provides synonyms of the English words whose Amharic translations are not in dictionary. this paper, maximum of four N-grams has been used. Most phrases of English that are translated to single word in Amharic have a length of four or less words.
For example the English phrase "everyone who calls the name of the lord will be saved" has the translations in Output using our dictionary.
everyone', "065" ne ('everyone', "on"], d'vveroone', 'AD*?"),
"everyone', 'mp55a7 n='", ['everyone', 'ANF3934'1] ']], ['who', '''''],,
['calls', 'T<P'', ['calls', 'r"Lma'], ['calls', 'hnsm']],
['on', 'As']l,
['name', 'm0"', ['name', 'hm'], ['name', 'nam"", ['name', 'AnFum]],,
[['o'' 'r']l.
['Lord', '22'], ['Lord', 'russ',, ['Lord', 'zaF'',, ['Lord', 'ma=''),
'Lord', 'm=#5''], ['Lord', '2m''], ] ['Lord', 'wiear], ['Lord', 'Aza'], "Lord', 're='], ('Lord', 'm='], ['Lord', 'r2#F1s'), ['Lord', '2#73'' "Lord', 'rr#Fs']],
['will', 'd.9x']]. ['will be saved", 'sesas']], [[be', "mmrs''
['saved', 'sesav'], ['saved', 'As8s>'), ['saved', 'mm4"FF''), ['saved', "eessa"" ", 'saved', 'sæsa'], ['saved', 'Amaæ*'], ('saved", "Huesuu',, [' ['saved', "aaavedd/'""tteeee /1mterrm=""" ', 'saved', 'sRss']]]
Output 1: The translated output of the N-grams
These translations have been combined into sentences in the same order as the source sentence. Then each sentence is converted into N-grams of variable length. The maximum flooded N-grams length (en(
if len(NN (engaa) 24 else equal to len(Ngram). This provides a good variable range to capture neighboring words in single N-gram.
2
3

---

Under review as a conference paper at ICLR 2020
Output shows sentences formed using the translations in Output [ The N-grams to be flooded formed by sliding one word from left to right through the combined sentence.
To nw- T6P4 no "Ld B.RAA,, "0 (w- TeP no el BREGAS 'u car pmnLmdd t19h LL SEEAAS" "u nan Lmd ho PlAn Besas" 'i TEP n1 "Lsh SEGAY, "i TEP hi Fl BRGAS, 'un Lmd, n1 "Lth- ERGAN, '0h s"Rme h19 Fl BEGAN"]
Output 2: The translated output combined into sentence
Output 3 shows the N-grams for the translated sentences shown in Output 2
['UNP na- TePt np,", "oNt"" ne- PaLme tt"", 'wh TEPT n5", 'oh r'mme. nr"),, ('hm Tep' to rt', 'na Tept rles', yoandd "h",, roLme n7 "Les', "neP" or "h', 'mep' PLA'', 'poZméd "La 'p"Lmn to rum''l, ('rep' to Besas', 'TeP' tom Besas', 'p"mme BESaS', 'r"mmdd to PlAs sesas', ".m *nt BReaS', 'trp Plts [esas']]
Output The N-grams for the translated sentences
The flooder is then responsible for searching the translated phrases in the target corpus and finding the longest N-gram match. For each phrase be flooded, selects phrase in the target corpus with the most translated words and least in- between words amongst the words matched. The flooder produces the result in Output with the Book of Romans as the target corpus to flooded.
Illl'mm' "mna', Tax', "MF",, "ARR', "H'', "ow', TTF' 'mraa''' "ihaa J HHAA', 'mL'' 'm926" 'm'' 'ax', 'neF', 'on', 'mm'' 'panper" "oar",, { ('nm" 'r"mm""), [' 'ramee', 'on'') '],, [l'r'', 'ne', 'papercnnt' 'nw'), ['ram'' 'nr"), ['nme, 'rran'', ('ree'//nr' 'raan')] }], [['m, 'paane', "on', 'sesas"), ['rass', 'amm, 'rane', 'ea", 's]sas']]])
Output Final output of flooder for single flooded file
The N-gram connector combines the flooded text to find the longest overlap of the translated target text. The overlapping system favors those with the least number of not searched words found in between the searched N-grams when calculating the overlap. Output S shows the final outcome of the N-gram connector.
The system selects the maximum longest overlapping phrases from the combiner and merges them to form the final target sentence. So finally, the translation for the example English phrase "everyone who calls the name of the lord will be saved" is rran to mndd of BRSAF..
3.2 NMT SYSTEM
In this paper, have used RNN (recurrent neural networks) for the NMT. In RNN, the output fed back to the neuron to learn from both the fresh input and its previous output. This improves RNN's performance because learns from its emrors while training.
The neural cell used is the LSTM (long short-term memory), introduced by Hochreiter & Schmid huber (1997). We have used LSTM cells for both the encoding and decoding of the sentences. For the decoding, a greedy algorithm has been used. The algorithm selects the first fit word that has the highest probability of occurrence. Probability refers to the probability of being translated and appearing next the word before itself.
The system has an attention layer between the encoder layer and the decoder layer. We have used the Luong attention model (Luong al. 2015). Equation [hroughh Equation 4 shows Luong attention model's 's computation.
exp(score(hb,, h.))
O11a
[Attention Weights]
(I)
ezp(score(hs. F)))))
4

---

Under review as a conference paper at ICLR 2020
[1' 'poLnd'), I, (' 'r"ind" Il,(('rs''' 'np", 'preme'), ['Pr0'', 'nFe ppeene', '00" "Besas' [Ill'rao'' 'n0" 'rrand' 'oh', "BesAs'], 1, ['r20z', 'nte', 'rame'', 'on" BBESAS
Output 5: Final Output of N-gram connector
C a= a= Lashhs
[Context vector]
f(c, he) - tonh(Walce h.]) [Attention Vector] score(ha, h() = 11 WT. [Luong's multiplicative style]
(4)
The score function, calculated using Equation is used to compare the output of the decoder (h,)
with the output of the encoder (h2) in order to find the attention weight calculated using Equation 1
The Attention weights (alpha;s) are then used for the context vector(c,) calculated by Equation 2
This context vector well the output of the decoder is then used produce the final output of the decoder using Equation 3
3.3 COMBINATION OF CBMT AND NMT
To combine the two systems, we have made the NMT model to accept two inputs. We have used the proposed method of combining PBMT with NMT accepting two source inputs by Zoph & Knight (016)_ According to Zoph & Knight (2066), having two inputs, where one is the source sentence and the other translation the source another language different from the target, helps the NMT produce better result.
The source sentence and the sentence translated using the CBMT are encoded separately and are given to the attention layer. The attention layer focuses on the two inputs at the same time rather than separately. There is a single decoder, which receives the output of the attention layer and provides the final translation.
Based on the paper Zoph & Knight 2016). the final outputs of the encoders (h₂) are concatenated and linear transformation is applied the concatenated output which is activated by tanh using Equation [5] On the other hand, the final states (c₂) are simply added as shown by Equation o In the attention, the different context vectors are calculated separately and concatenated to produce
final output of the attention vector based on the Luong attention mechanism (Luong al. 2015)
using Equation]
tanh (Wa((WWa
(5)
c C1 + 02
(6)
h tonh(w.lhhsc
(7)
4 EXPERIMENT SETUP
We evaluate the proposed approach for the language pair English-Ambaric using the same training parameters for both the basic NMT and the combinational system. The encoder-decoder setup has 1024 LSTM cells hidden units with 1024 word embedding and the data has been trained for 100 epochs.
4.1 CORPUS USED
This research uses the new International Version Bible because both the Amharic and English ver- sions are translated from the same Dead Sea scrolls. This makes it more accurately parallel than other versions of the Bible translation.
The whole New Testament of the Bible has been used as corpus providing total of 8603 phrases and sentences. We have used two books of the Bible, Paul's letter to the Romans (Romans) and the Gospel according to Mark (Mark), as test set.
Google translate has been used as the main agent of translation for the manually built bilingual dictionary used by the CBMT. 77% of the total 6,793 vocabulary words have been translated using
5

---

Under review as a conference paper at ICLR 2020
Google. In addition to Google translate; have done manual translation for 1,426 vocabulary words in the book of Romans and other 150 vocabulary words using the Bible.
Manual translation here refers to the translation of cach word using every entry it has in the Bible by a human. Figure 5 shows the outcome of such translation for the word Acknowledge. Manual translation helps to address the variations in Amharic translated words caused by gender (female or male), plural form and the different persons (first, second and third persons). English words do also have different Amharic translations based on their context as shown in Figure 3 been translated into four main stem words A04, +40 and onhh..
Acknowledge has
acknowledge hm-25 BanthC A98:A/12222
Mhn
A
fot
Figure 3: Manual translation result of the word acknowledge
4.1.1 DATASET
We have fed the same dataset to all systems with minor variations. In the CBMT, we have used the book of Mark and the book of Romans test set. The flooded texts for the book of Romans were the book of Romans itself and Paulian epistles without Romans. The flooded texts for the book of Mark were the book of Mark itself and the gospels without Mark. The books have been flooded to themselves in order to evaluate the performance of the CBMT when the searched text is found in the flooded text and also to see the impact of the bilingual dictionary on the CBMT.
The combinational system has two different test sets and different models. The first test set has the output of the CBMT and the source sentence as an input for the NMT. The second test gives the original target text and the source sentence as an input to the NMT models and we have called it the ideal approach. This was done so to see the impact of the CBMT output and the errors introduced by the CBMT on the combinational system.
In the basic NMT and combinational system, similar dataset as the CBMT is used. We have used Paulian epistle without Romans to train basic NMT model and the combinational model. Then we have tested the model using the book of Romans as the test or holdout set. We have used 10- fold cross validation(Diettrrich, 1998) to train and test the basic NMT model and the combinational model with Romans. In a similar manner, we have used 10-fold cross validation Dietterrci 1998) to train the basic NMT model and the combinational model with the book of Mark. We have also used holdout SindattionRaschkee 2018) with a random 80% training and 20% test data split alongside the 10-fold cross validation for both Mark and Romans to obtain more general representation of the results.
5 EVALUATION METHOD
We have measured the translation performance based on the fullness of the translation, whether the system translates all words; context awareness, whether the translation is true to the context and Coherence, whether the translated sentence has smooth flow of words in terms of syntax.
In this research, the BLEU score is the chosen method of evaluation. answers all the above- enlisted criteria. The quality of translation is measured by the correspondence between a machine's output and that of human(Kishore et al. 2002). BLEU score is defined in the range between 0 and I (or in percentage between 0 and 100) where I is a perfect match with the reference and 0 is for no words matched.
6 RESULTS AND DISCUSSION
This section provides the results obtained along with a brief explanation of the factors and the com- ponents.

---

Under review as a conference paper at ICLR 2020
6.1 CBMT RESULT AND DISCUSSION
The system has been tested using a custom-made dictionary using Google translate and manual translation. We have generated the vocabulary of the dictionary from the English version of the NIV Bible. Table T depicts the CBMT test results obtained, using BLEU score evaluation method. We
Table 1: Results for the CBMT
Flooded Data
Test data
BLEU score
Paulian epistle without Romans
Romans
27.91
2 Romans
Romans
70.46
3
Mark
Mark
21.98
Gospels without Mark
Mark
21.43
have implemented manual translation for the book Romans on about 80% of its total vocabulary. Hence, it has better performance yield than the book of Mark, whose translation was solely de- pendent on Google translate. This is so both when they are flooded to the text that contained them (by 48%) and when they flooded to the text without them (by 6%). However, the translation of Romans does not produce a 100% as would be expected when it is part of the flooded document. This is mainly because the system selects the overlapping N-gram based on the number of words matched, two consecutive phrases that may have high overlap but which are not the correct ones may be selected.
6.2
NMT RESULT AND DISCUSSION
The NMT test results obtained using BLEU score evaluation method are depicted in Table 2 The
Table 2: Results for the NMT
Training input Data
Test data
Validation
BLEU
Paulian epistle without Romans
Romans
Holdout
10.24
Romans
Romans
10-fold
11.95
Mark
Mark
10-fold
12.42
Romans
Romans
Holdout
7.28
Mark
Mark
Holdout
10.12
test result obtained from Mark was better than that from Romans by average of 1.62%. Although insignificant difference, attributes to Marks' writing having similar words unlike the diverse word selection in Romans(Clay 2018).
6.3 CBMT AND NMT COMBINATION RESULT AND DISCUSSION
There are two test cases for this section. In the first case, we have given the NMT the source sentence and the output the CBMT as an input per the proposed methodology. Table 3 shows the results obtained from such setup. In the second case, we have given the NMT the English source sentence
Table 3: Results for the combination of NMT and CBMT
Training input Data
Test data
Validation
BLEU
Paulian epistle without Romans
CBMT output Romans
Holdout
11.55
2
Romans
CBMT output Romans
10-fold
14.07
Mark
CBMT output Mark
10-fold
12.36
Romans
CBMT output Romans
Holdout
13.84
Mark
CBMT output Mark
Holdout
12.73
and the original Amharic input creating ideal system. The test results obtained using BLEU score evaluation method, are depicted Table the ideal combinational system. In the first case, when the CBMT output is used as input to the NMT, the Book of Romans performed better than

---

Under review as a conference paper at ICLR 2020
Table 4: Ideal case Results for the combination of NMT and CBMT
Training input Data
Test data
Validation
BLEU score
Paulian epistle without Romans
original Amharic Romans
Holdout
11.63
2
Romans
original Amharie Romans
10-fold
18.46
3
Mark
original Amharic Mark
10-fold
17.41
5
Romans
original Amharic Romans
Holdout
25.74
Mark
original Amharic Mark
Holdout
25.52
the book of Mark by 1.71%. The CBMT output of Romans better than that by the book of Mark and its impact has propagated to the combinational system. In the ideal case scenario the results is more or less the same. The result for the book of Romans was better than the book of Mark by only 0.63%.
6.4 DISCUSSION OF ALL RESULTS
The average of the results obtained from the systems have been calculated and shown in Tables 5 for comparison. The ideal combinational system, which takes the original target Amharic text as the
Table 5: Summary of all results
System
Dataset
BLEU
book Romans
27.91
CBMT
book of Mark
21.43
book of Romans
9.61
simple NMT
book of Mark
11.27
book of Romans
13.95
Combinational system with CBMT output
book of Mark
12.54
book of Romans
22.1
Combinational system with original text (idcal)
book Mark
21.46
second input, has performed better on average with 11.34 BLEU score gain over the NMT. The ideal system, however, did not outperform the CBMT average but produced results in the same range. The combinational system with CBMT output given as the second input for the NMT, achieves 2.805 BLBU score point the simple NMT. The CBMT without being provided the target flooded data has performed better by 14.23 BLEU points over the simple NMT.
7 CONCLUSION
The research set out to find a combinational system with components that complement each other when translating document from English to Amharic. We have proposed the CBMT and the NMT as the complementing pair in terms of parallel data size, accuracy, context awareness with translation and coherence.
The CBMT system performed better than the basic NMT and the combinational system given the same size of data. However, the ideal combination of the CBMT and NMT has BLEU score in the range of the CBMT while outperforming the simple NMT by 11 11.34 BLEU points. For the book of Mark whose writing resembles the other Gospels, the ideal combination outperformed all. This entails that with smaller increase in parallel corpus for the NMT of the ideal system will outperform both individual systems. The output from the CBMT has a great impact on the performance of the combinational systems seen by the performance of the proposed combinational system compared to the ideal system. The proposed combinational system still has outperformed the basic NMT by 2.805 BLEU score points in spite of the errors introduced by the CBMT.
Therefore, CBMT with a well- built bilingual dictionary that produces a close to ideal output along with well-trainel NMT with sufficient data makes fluent combinational system that outperforms a simple NMT and a basic CBMT system. In conclusion, the study suggests the combinational system for translation of English Amharic language.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. Unsupervised neural machine translation. CoRR, abs/1710.11041, 2017. adtt:://arzi...zg/abbb.
Laurent Besacier, Michael Melese, and Million Meshesha. Amharic speech recognition for speech translation in tourism domain. 2000.
Peter F. Brown, John Cocke, Stephen A. Della Pietra, Vincent J. Della Pietra, Fredrick Jelinek,
John D. Lafferty, Robert L. Mercer, and Paul S. Roossin. statistical approach to machine translation. Computational Linguistics Volume 16 Number 2, 1990.
Cholee
Clay.
Comparing the gospels: Matthew mark, luke, and
john, 2018.
URL https://owlcation cow/humanities/
Compar Lng---ge the Gospel Matthew Mar-- and- John
Thomas G. Dietterich. Approximate statistical tests for comparing supervised classification learning algorithms. Neural computation, 1998.
Rashmi Gangadharaiah. Coping with data sparsity in example based machine translation. 2011.
Michael Gasser. Toward rule-based system for english -amharic translation. 2012.
Jiatao Gu, Hany Hassan, Jacob Devlin, and Victor O. K. Li. Universal neural machine translation for extremely low resource languages. CoRR, abs/1802.00 802.05368, 2018. URL http://arxiv.
org/abs/ /10...110...0 05368
Sepp Hochreiter and Jurgen Schmidhuber. Long short-term memory. 9:1735-80, 1997.
Papineni Kishore, Roukos Salim, Ward Todd, and Zhu Wei-Jing. Blue: a method for automatic evaluation of machine translation. Association for Computational Linguisties, pp. 311-318, 2002.
Gorka Labaka, Cristina Espaa-Bonne Bonet, Llus Mrquez i Villodre, and Kepa Sarasola. A hybrid machine translation architecture guided by syntax. machine translation, 2014.
Minh-Thang Luong. Hieu Pham, and Christopher D. Manning. Effective approaches to attention- based neural machine anslation.
2015.
David Miller, Jaime Carbonell, Steve Klein, Mike Steinbaum, and Tomer Grassiany. Context-based machine translation. The Association for Machine Translation of the Americas (AMTA-2006), 2006.
ThanhLe Ha Jan Nichues, Eunah Cho, and AlexWaibel Pre-translation for neural machine transla-
tion. 2016.
John Oladosu, Adebimpe Esan, Ibrahim Adeyanju, Benjamin Adegoke, Olatayo Olaniyan, and Bo- laji Omodunbi. Approaches to machine translation: review. FUOYE Journal of Engineering and Technology. 1:120-126, 09 2016.
Maja Popovic. Comparing language related issues for nmt and pbmt between german and english. The Prague Bulletin of Mathematical Linguistics No. 108, 2017, pp. 20922, 2017.
Sebastian Raschka. Model evaluation, model selection, and algorithm selection machine learning, 2018.
Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pp. 1715-1725, Berlin, Germany, August 2016. Association for Computational Linguistics. doi: 10. 10.18653/w1/P16-111 orte/anthhoLogy//I1-111
URL https: www aclweb.
Dagobert Soergel. Wordnet. an electronic lexical database. 10 1998.
Ambaye Tadesse and Yared Mekuria. English to amharic machine translation using smt. Master's thesis, Addis Ababa University, 2000.

---

Neural system combination for Under review as a conference paper at ICLR 2020
Girma Taye, Mulu Gebreegziabher, Laurent Besacier, and Dereje Teferi. Phoneme based english- amharic statistical machine translation. AFRICON 2015, pp. 1-5, 2015.
Eleni Teshome. Bidirectional english-amharic machine translation: An experiment using con- strained corpus. Master's thesis, Addis Ababa University, 2000.
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoe Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey. Jeff Klingner, Apurva Shah, Melvin John- son, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Gregory S. Corrado, Macduff Hughes, and Jeffrey Dean. Google's neural machine translation system: Bridging the gap between human and machine translation. CoRR, abs/1609908 144, 2016.
Evi Yulianti, Indra Budi, Achmad N. Hidayanto, Hisar M. Manurung, and Mirna Adriani. Devel- oping indonesian-english hybrid machine translation system. 201 International Conference on Advanced Computer Science and Information Systems, 2011.
Samrawit Zewgneh. English-amharic amharic document translation using hybrid approach. Master's 's thesis, Addis Ababa University, 2017.
Jiajun Zhang, Long Zhou, Wenpeng Hu, and Chengqing Zong. machine translation. In ACL, 2017.
Barret Zoph and Kevin Knight. Multi-source neural translation. Cl., 2016.
10


Reviews:
[{"review_id": "r1gCkziQcr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a machine translation system based on a combination of a neural machine translation system (NMT) and a context-based machine translation (CBMT). The method is evaluated on a small parallel corpus application of English-Amharic translation. The idea is that in the small corpus setting, the CBMT can leverage a manually built bilingual dictionary\nto improve on the standard NMT.  \n\nClarity: The paper is reasonably clear, though there are numerous typos and minor language glitches (missing articles, etc.). Overall the quality of the writing is probably a bit below what's acceptable for publication at ICLR, but nothing that could not be fixed on subsequent revisions.\n\nNovelty: The method proposed appears to be a fairly straightforward variant of one proposed in a previous paper, where an NMT system was combined with a phrase-based MT system (Zoph & Knight, 2016). There seems to be no novel machine learning contribution (nor is it claimed). This paper seems more appropriate for a venue more focused on machine translation rather than a machine learning venue such as ICLR.\n\nEmpirical evidence in support of the claims: The authors set out to demonstrate that by combining a CBMT output into an NMT approach, one can get the best of both approaches. Their results do not strongly support this claim. The results suggest that in the context of the small-scale experiments considered, the baseline CBMT model is actually overall the best performing model. It is therefore strange that, in their last sentence of the conclusion, the authors persist in claiming that their combination \"outperforms a simple NMT and a basic CBMT system\". That being said, the sub-claim that the\nNMT/CBMT hybrid improves on the baseline NMT system is well established. \n\nIn light of the relatively low novelty and the lack of compelling empirical performance for the proposed combined MT system, I do not feel that this paper is appropriate for ICLR at this time.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "r1e-hTxAKH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "For the low-resource pair English-Amharic, the authors propose to combine context-based machine translation (CNMT), which is built by using a bilingual dictionary and then connecting the resulting n-grams, with a neural MT system. The source sentence, as well as the CNMT output, are used as inputs to the RNN-based translation system.\n\nI vote for rejection because the paper makes some unfounded claims, misses important related work, has some methodological issues and presents unconvincing results.\n\nThe paper claims that CBMT is a new approach, but it dates from 2006. The authors say that it outperforms other MT approaches, but a more recent reference would be needed. While neural machine translation may sometimes struggle with rare words, using sub-word units may help alleviate the issue (Sennrich et al. Neural machine translation of rare words with subword units). The claim that RNNs learn from their previous mistakes is also unclear. It's true in the sense that backpropagation is learning from errors, but using previous reference outputs can cause exposure bias (Ranzato et al. Sequence Level Training with Recurrent Neural Networks).\n\nThe paper fails to cite related work, in particular on low-resource NMT (e.g. Gu et al. Universal Neural Machine Translation for Extremely Low Resource Languages) and unsupervised translation (e.g. Artetxe et al. Unsupervised Neural Machine Translation).\n\nThe CBMT system is built on top of the Google Translate English-Amharic system. However, that model may have seen the test data during training.\n\nBy combining CBMT with NMT, the authors obtain better results than NMT alone, but worse than with CBMT only. As such, the usefulness of the approach in a very low-resource scenario is unclear.\n\nMinor points:\n\nSome typos could be corrected: BLUE -> BLEU, Loung -> Luong, weather -> whether", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "Hyl4Le2g9r", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper aims to combine a traditional CBMT system with an NMT system. The core idea of the paper is to use the output of the CBMT system as a second source to a multi-source NMT system. The first source of the system is CBMT, the second source is the original source and the output is the translation in the target language. All the experiments are conducted for English-Amharic with a small amount of parallel data from the Bible. A lot of details are provided about the generation of the CBMT system using Google Translate and details are provided about the approach to create such a system.\n\nPros:\n- The idea of using additional outputs to the NMT system and outputs from a context-aware system is neat. This has been done by others who have included PBMT outputs in the past. However, this might be the first to include a CBMT result as an additional source.\n- Focusing on a low-resource language like Amharic is good for the community and will encourage more research in these underrepresented languages.\n\nCons:\n- A lot of the techniques described for building the traditional CBMT system are obsolete these days and people prefer neural methods. I worry if this is relevant in the current day.\n- The authors could have compared against other ways of incorporating context as a strong baseline - like a n-to-n or n-to-1 NMT system.\n- Most experiments in the paper are conducted on a small data set and this is a big downside of the paper.\n- More detailed analyses of where the contextual phenomena was incorporated might have helped the paper.\n\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors propose a model which combines a neural machine translation system and a context-based machine translation model, which combines some aspects of rule and example based MT.  This paper presents work based on obsolete techniques, has relatively low novelty, has problematic experimental design and lacks compelling performance improvements. The authors rebutted some of the reviewers claims, but did not convince them to change their scores. 
Decision: Reject