Transition Based Dependency Parser for Amharic Language Using Deep Learning
Authors: Mizanu Zelalem
Million Meshesha (PhD)
Venue:  (2020)
Keywords: Amharic dependency parsing
arc-eager transition
LSTM
Transition action prediction
Relationship type prediction

Abstract: Researches shows that attempts done to apply existing dependency parser on morphological rich languages including Amharic shows a poor performance. In this study, a dependency parser for Amharic language is implemented using arc-eager transition system and LSTM network. The study introduced another way of building labeled dependency structure by using a separate network model to predict dependency relation. This helps the number of classes to decrease from 2n+2 into n, where n is the number of relationship types in the language and increases the number of examples for each class in the data set. Evaluation of the parser model results 91.54 and 81.4 unlabeled and labeled attachment score respectively. The major challenge in this study was the decrease of the accuracy of labeled attachment score. This is mainly due to the size and quality of the tree-bank available for Amharic language.  Improving the tree-bank by increasing the size and by adding morphological information can make the performance of parser better.

Under review as a conference paper at ICLR 2020
TRANSITION BASED DEPENDENCY PARSER FOR AMHARIC LANGUAGE USING DEEP LEARNING
Anonymous authors Paper under double-blind review
ABSTRACT
Researches shows that attempts done to apply existing dependency parser on mor- phological rich languages including Amharic shows poor performance. In this study, a dependency parser for Amharic language implemented using arc-eager transition system and LSTM network. The study introduced another way of build- ing labeled dependency structure by using a separate network model to predict dependency relation. This helps the number of classes decrease from 2n+2 into n, where n is the number of relationship types in the language and increases the number of examples for each class in the data set. Evaluation of the parser model results 91.54 and 81.4 unlabeled and labeled attachment score respectively. The major challenge in this study was the decrease of the accuracy of labeled attach- ment score. This is mainly due to the size and quality of the tree-bant available for Amharic language. Improving the tree-bank by increasing the size and by adding morphological information can make the performance of parser better.
INTRODUCTION
Amharic is one of the Ethiopian languages, grouped under Semitic branch of Afro-Asiatic language. Amharic serves the official working language of the Federal Democratic Republic of Ethiopia and it is the second most spoken Semitic language in the world after Arabic with a total speak- of around 22 million, as per the census of 2007Asker (2010). Even though many works have been done to develop
tools, they are mainly concentrated on English, Euro- pean and East-Asian language=(Nivre 2013).In spite of large number of speakers, Amharic one of the under-resourees languages in that it needs many linguistic tools to be developed in general and dependency parser in particular. Researches like Maltparser attempted to develop universal depen- dency parser that can be used to parse sentences from different languages without making language specific modification to the system F.] (2007). Even though it is language independent dependency parser, few languages are incorporated in to Malt parser, such as English, Dutch, Turkish and Ital- aumE.)(200)) Parsing in morphologically rich languages is challenging. This is because of the fact that, in morphologically rich languages, dependency relation exists between not only orthographic words (space-delimited tokens) but also relation within the word Nivre (203)). For example, in Amharic, orthographic word combines some syntactic words into one compact string. These words can be function words such as prepositions, conjunctions and articles. This makes an ortho- graphic word functioning as phrase, clause or sentence(innam Ephrem Seyoum| (2018). Studies on applying existing parser system on morphologically rich languages were conducted in Aske (2010) and marton; Nizar Habash; Owen Ranbow (2012). these studies showed applying parser like MST and malt parser on morphological rich languages results a poor performance. The first attempt to develop a dependency parser for Amharic language was made in Gasser (2010). Gasser used Exten- sible dependency grammar (XDG) to accomplish the work. However, their system was not tested on corpus data and that makes the system inapplicable. This aim of this study is to design and develop dependency parser for Amharic language that uses the rule of arc-eager transition system. The main contribution of this paper are: (i) designing a neural network model with LSTM and attention mech- anism that gives good accuracy, (ii) introducing separate model for building labeled dependency structure, (ii) developing a dependency parser that can produce unlabeled and labeled dependency structure for Amharic sentences.

---

Under review as a conference paper at ICLR 2020
1.1 DEPENDENCY PARSING
Dependency parsing is one of the techniques for analyzing syntactic structure of a sentence. It represents the head-dependent relationship between words in a sentence classified by functional cat- egories or relationship label (2015). The advantage of dependency parsing is that it provides a clear predicate argument structure, which is useful in many NL.P applications that makes use of syntactic parsing.Nivre (2010). Approaches to perform dependency parsing are broadly grouped into grammar based and data driven approache<Nivre (2010). In grammar- approach, constructing dependency structure rely on explicitly defined formal grammar. In such approaches, parsing is defined as the analysis of a sentence with respect to the given grammarNivve proaches for dependency parsing uses statistical method to build the dependency structure. In such approaches, the parser is trained on labeled dataset to learn the pattern of dependency relation, latter it uses its memory parse new sentences Ninve (2010). Data driven dependency parser is further grouped in to transition based and graph based approaches. In transition-based dependency parsing, the parser uses transition systems to build a dependency structure. In this method, statistical model is trained on sequences of transition configurations or states for predicting transition action for new Nive (2003). Then the parser uses the predicted sequence transition actions guideline to build dependency structure for given sentenceNivee (0055).
(OTO).Date driven ap-
1.2 AMHARIC LANGUAGE
Amharic is a Semitic language spoken in Ethiopia. It is the second wide mostly spoken Semitic language in the world next to Arabic Asser (2010). Amharic exhibits a root-pattern morphologi- cal phenomenon in which the root is combined with particular prefix or suffix to create a single grammatical form or another sten/Demeke & Getachew (2016) Asker (2010). Because of this, dif- ferent authors use different number of tag sets to incorporate the created word forms. Some authors use a different tag sets for each of the morphological word:Demeke & Getachew (2016) and other separates the clitic from the host word and assign part of speech tag to host word and the clitic sep- arately Binyam Ephrem Seyoum (018)). We used a treebank from provided in inBinnam Ephrem Sey- oum 2018), for training and constructing network model. This treebank has 23 tag sets (see table 1) and uses the second method of assigning part of speech tag.
2 METHODOLOGY
The proposed Amharic dependency parser consists of two phases, which are training learning phase and parsing phase, see figure 1. In the first phase, deep learning models are trained on Amharic treebank to learn the pattern of Amharic dependency relation. We divide this task in to two steps. The first step is training transition action predictor. This step is used to learn the pattern of arc-cager transition action Amharic treebank. Once arc-eager transition predictor model is constructed, also train another model for predicting dependency relationship type. The second phase is parsing phase in which the trained deep learning models are used for constructing dependency structure to an Amharic sentence.
2.1 LEARNING RELATIONSHIPP TYPES
In this step, a network model, see figure 2, is trained on Amharic treebank to learn the pattern of arc-eager transition actions For doing this we convert Amharic treebank in sequence of arc-eager transition configurations. A single configuration have five elements (stack state, stack part of speech tag state, buffer state. buffer part of speech tag state, and transition action), see figure 3. Each inputs then embedded to make a dense representation. That means words that are similar or used in similar grammatical places will have a closer numerical value. For example, the word 'werewwre' and gez' will have a closer weight than other words because they have similar grammatical place, in this case, they are used as a verb in the sentence. After each inputs are embedded, related inputs (stack of words state with stack of words POS state and buffer of words state with buffer of words POS state) are scalar multiplied using equation and equation
P= W. T Q= ..NN6. N.......... S...............................
(1)
(2)
Where W is words in the stack, T is the corresponding part of speech tags the stack, N is words in
2

---

Under review as a conference paper at ICLR 2020
Ambaric trebank
Converting to Sequence
Converting into Head-
Trained Network
Trained Netaookk
Network Predict Tranatiion action
Predicting
Training Predict Reta tonahip Types
Predicting Relatin Slnshhp
and Buffer State
actions
Predicting Relationship Types
Morphological anatysts and lagging
La beled dependency parse (output
Arceager tra frn algorithem
Unfabeled dependency (outport
Converting Dependent
Parsing phase
Figure I: Architecture of the parser
Stack states
Stack
Buffer state
Buffer POS state
Embedding layer
Embedding layer
Embedding layer
Embedding layer
Dot product
Dot product
LSTM layer
Concatenation
LSTM layer
LSTM layer
Attention layer
Time distributed Dense layer
Transition action
Figure 2: Network model of learning transition action
3

---

Under review as a conference paper at ICLR 2020
Stack state
Stack POS state
Buffer state
Buffer POS state
Transition action
[-root-]
[-root-]
[duZ. h, 3, 06 A , ::]
[NOUN, DET, ACC, VERB, SUBJC, OBJC, PUNCT]
Shift
[-root-]
[-root-]
[ERA, 371. A, NZ. #]
[NOUN, VERB, SUBJC, shift ADP, OBJC, PUNCT]
Figure 3: Sample instances of transition configuration
the buffer and S is the corresponding part of speech tags in the buffer. Before concatenating the stack state and buffer state, we used LSTM layer to make information to be connected from one-step to the next in the stack and buffer states. The outputs of the two LSTM layers (LSTM layer for stack state and LSTM layer for buffer state) are concatenated. The concatenation layer simply combines matrix of stack state and buffer state. The last LSTM layer then applied to the concatenated stack state and buffer state, which are separately passes thought LSTM layer. This helps the information further connected from stake state buffer state. Since arc-eager transition system keeps the word at stack before all its dependents at the buffer are seen, the transfer of information from stack state to buffer state helps the network to incorporate this phenomenon. The final layer of the network model is fully connected dense layer. It is applied to each time step of the outputs from the previous layer decide the final output. The output of the network is one of the arc-eager transition actions (shift, left-are, right-are or reduce).
2.2 LEARNING RELATIONSHIP TYPE
Studies on transition based dependency parsing constructs unlabeled and labeled dependency parse from given transition configuratiorl.. (2007 (2007 (2014) 2 That means they use 2n+2 classes during training the classifier, where "n' is the total number of relationship types in the lan- guage. This method has disadvantage that it increases the number of classes in the learning model so that the number of examples for each classes in the dataset will be small. This disadvantage can be even severing when the dataset smaller. Besides that, as we observed in the experiment, the relationship type that holds between dependent word and headword is not dependent on the transition configuration. That is because, transition configurations only give an information about which action to select among the four transition actions. On the other hand, the type of dependency relationship can be determined by using the part of speech tag of the dependent and headword. For example, the relationship type between and in the sentence "... is det because of the part of speech tag of the word is DET.
In this study, we designed a separate model, presented in figure 4, to learn the relationship type of head-dependent pairs in Amharic treebank. To do this we convert Amharic treebank into head- dependent pairs. The network model accepts part of speech tag of the dependent word and the part of speech tag of the corresponding headword. figure 5 shows sample of inputs to the network model For simplifying access to the network model, shown in figure 5, ROOT (a word that is exist in the original sentence) word is artificially added. This word is used to indicate the root word of a sentence during dependency construction.
Embedding layer is applied to the POS tag of dependent word and headword separately to represent in dense vector. The relationship between the dependent word " and the headword " is 'expl'. The relationship type 'expl' is used if the subject is explicitly indicated in the sentence, in our case, for instance the word ". The other condition the subject not explicitly indicated the sentence. An example is given in figre 6 above. In this case, the relationship between the dependent word and the headword " become 'nsubjc'. To handle this situation we applied LSTM layer. LSTM helps to transfer information from one time step the next. In our case, it helps to pass informa- tion about the presence of subject word in the previous steps to select appropriate relationship type, either 'nsubj" or 'expl', for the head-dependent pairs accordingly. Finally, output is determined
4

---

Under review as a conference paper at ICLR 2020
Input (Head dependent pair of Amharic treebank)
Part of speech tag of Dependent words
Part of speech of head words
Embedding layer
Embedding layer
Concatenat
LSTM layer
distributed dense layer
Output (Relationship type)
Figure 4: Network model for learning relationship type
Dependent
Dependent
Headword
Headword's s POS tag
Class (relationship
word
word's s POS tag
type)
KUU
PROPN
A
VERB
nsubi
00
NOUN
0A
VERB
obi
AA
VERB
ROOT
ROOT
root
SUBJC
AA
VERB
expl
Figure 5: Sample inputs for the dependency relation predictor network model
Dependent
Dependent
Headword
Headword's
Class (relationship
word
word's 's POS tag
POS tag
type)
NOUN
AA
VERB
obi
0A
VERB
ROOT
ROOT
root
A
SUBJC
nA
VERB
nsubi
Figure 6: Sample inputs for the dependency relation predictor network model
5

---

Under review as a conference paper at ICLR 2020
using time distributed dense layer. This layer outputs the predicted dependency label for given head- -eadddependent pairs of a sentence.
2.3 PARSING PHASE
The purpose of this phase is construct dependency relation for a given new sentence by the help of trained network models. the steps followed in parsing phase are shown in figure 1. Step 1: Input. The input for the parsing system is an Amharic sentence. For example " Step2: Morphological analysis and part of speech (POS) tagging. In this step, morphological analysis and POS tagging is performed on the input sentence. We perform this task manually using the method presented on UD-Amharic treebank.
Step 3: Initializing stack state and buffer state. The network model for predicting transition action accepts four values (stack state, stack POS state, buffer state and buffer POS state), Due to this, the imput sentence is used to initialize these states. The initial state of stack contains one element, 'root". This element is used later as headword for the root word of the sentence. The initial state of buffer contains all tokens of the input sentence. Stack POS state and buffer POS state contains corresponding POS tag of the words in the stack and the buffer respectively.
Step 4: Predicting transition actions. this step, the network model trained for predicting transition action uses the value of stack state and buffer state to predict the one of the four arc-eager transition actions for the configuration. After the model select the transition action, the algorithm updates the state of the stack and the buffer according to the transition action and the rule of arc-eager transition system.
Step 5: Arc-eager transition algorithm. In this step, arc-eager transition rule is used to build unlabeled dependency relation for the given input sentences. The sequence of transition actions in the previous step are used to guide the algorithm assert head-dependent relationship between the tokens and update the state of stack and buffer. When the buffer contains no element the algorithm stop the iteration and the constructed unlabeled dependency tree will be released.
Step 6: Converting into head-dependent pair. In this step, the unlabeled dependency parse is converted into head-dependent pairs to predict the dependency relation between the head-and dependent word.
Step 7: Predicting relationship type. In this step, the dependency relationship type of the head- dependent relation is predicted from the part of speech tag of head-dependent pairs using the network model trained for predicting dependency relationship type. The output of this step is labeled dependency structure of the given sentence.
EXPERIMENTAL RESULT
3.1 DATASET
In this study, we used the only Amharic treebank that is made available Brniaam Ephrem Seyoum 2018). It has 1074 sentences and we used additional 500 sentences that are manually constructed by the researcher with the help of linguistic experts at Jimma University.
3.2 EVALUATION OF TRANSITION ACTION PREDICTION
We experimented on 60/40, 70/30 and 80/20 train-test splitting ratios and achieved 93.1, 94.7 and 92.3 testing accuracy respectively for predicting transition action as shown in figure 8. Therefore, we select the 70/30 train-test splitting ratio throughout the experiments.
When we convert the treebank to transition configuration we got 26,242 configurations, among this we used 70 (with 1,102 sentences having 17,888 configurations) for training and 30 (with 472 sentences having 8,353 configurations) for testing. The dataset has 1475 unique words. We add other key words, such as -OOV- to indicate out of vocabulary words; -PAD- to indicate padding (zero) and 'root' to indicate the root word. Due to this, we got 1478 unique words. This makes the embedding layer for stack state and buffer state to have dimension of 1478. On the other hand, the dataset has 23 unique part of speech tags. Here also, we added -00V-, -PAD- and root key words and we have 26 unique key words for POS. This makes the embedding layer for POS state

---

Under review as a conference paper at ICLR 2020
59.00%
98.00%
57.00%
96.53%
96.00%
54.00%
93.100%
$3.00%
92.50%
52.00%
91.00%
90.00%
70/30
G0/40
80/20
Testing BCCuraCy
Figure 7: Accuracy of predicting transition action using different splitting ratio
90%
55.00
94.00%
53.00%
92.00%
61. 11.50%%
21.00%
000
attention
Figure 8: The accuracy of transition action prediction with and without adding of attention layer
of the stack and buffer to have 26 dimensions. For the LSTM layer we used 256 output dimensions and 128 dimensions for attention layer. For the output layer used four output dimensions that is 0, 1, 2 and 3 (for Shit, Left-are, Right-are and Reduce respectively). We used Adam optimization technique with learning rate 0.01 and batch-size 256. addition, the training iterates for 50 epochs. The experiment is conducted on two models, with and without adding attention layer. The result of the model to select the correct transition action for given configuration is presented figure 9. As shown in figure 9, the addition of attention layer helps the transition action prediction to achieve accuracy 94.7. Whereas, the accuracy of transition action prediction with no attention layer 91.5.
3.3 EVALUATION OF RELATIONSHIP TYPE PREDICTION
This experiment also used similar data the pervious experiment converted into list of head- dependent pairs. When the dataset is converted in to head dependent pairs, we got 15,534 head- dependent pairs. Among this, 70 (with 10,874 head dependent pairs) are used for training the model and 30 (with 4,660 head-dependent pairs) are used for testing the model. We used the embedding layer and LSTM layers 256-output dimension. We used 36 relationship types including 'root' (to indicate the root word) and 'PADD'' (to indicate padded zero) for the output dimension. In this periment, we use Adam optimization to train the network model with learning rate 0.01, 100 epochs and 256 batch-size. Out of 4660 head-dependent pairs, the model correctly predicts dependency relation for 3,839 (81.4) head-dependent pairs. The result shows the accuracy relationship type prediction if lower than that of transition action prediction. This due the limited size of Amharic treebank. The number of examples for relationship type prediction (with 36 classes) is less than the number of examples for transition action prediction (with four classes) in Amharic treebank.
3.4 ATTACHMENT SCORE
Unlabeled attachment score is the accuracy of the system to attach the correct headword with the dependent word without considering the relationship type. On the other hand, labeled attachment
7

---

Under review as a conference paper at ICLR 2020
Head-dependent pairs 4,660
Correctly
Scores
attached 4,266 pairs
percent
Unlabeled attachment score (UAS) of the
91.54 %
system
Labeled attachment 4,660 score (LAS) of the system
3,794 pairs
81.4
Figure 9: Attachment scores of the parser
score is the accuracy of the system to attach the correct headword to the correct dependent word and giving the correct relationship type. The parser is evaluated 4,660 head dependent pairs, which is 30 of the total dataset. As depicted in figure 10 the parser scores 91.54 accuracy for unlabeled attachment and 81.4 accuracy for labeled attachment score.
DISCUSSION
The study attempts to design Dependency parser for Amharic language. For the purpose two models are designed, transition action prediction and relationship type prediction model. For transition action prediction, two models evaluated. The first model is network model with no addition of attention layer. This model correctly predicts 91.5 transition actions of the test configuration from Amharic treebank. On the other hand, the second model uses addition of attention layer. This system correctly predicts 94.7 of transition actions from the test configuration. From this experiment, found that the of attention layer on the network model increases the performance of the prediction model by 3.2. The performance of relationship type prediction was also evaluated by using test datasets from Amharic treebank and correctly predicts 82.4 of the dataset. From this experiment, we found that the accuracy of the relationship type prediction decreased relative to accuracy of transition action. This is because of the size of the dataset. Relationship type predictor have 36 classes to be predicted. This makes the number of examples for each classes in Amharic treebank be smaller than the number of examples for transition action predictor, which have four classes. Due to this, the accuracy of transition action prediction becomes greater than accuracy of relationship type prediction. The system is evaluated using unlabeled attachment score (UAS) and labeled attachment score (LAS). From the experiment, the system correctly constructs 91.54 and 81.4 unlabeled and labeled attachments respectively. The challenge observed during in this study was the accuracy of the labeled attachment score. This is mainly due two reasons. The first reason the accuracy of the relationship type prediction is lower relative to accuracy of transition action prediction. This is further due to the size of Amharic treebank. The other reason is due architecture of the system. As described in section 6.3, the output of unlabeled dependency relation is passed the relationship type (dependency relation) predictor model wrong unlabeled tree constructed in the previous step makes the predictor model to choose wrong dependency relation.
input. Because of this, the
CONCLUSION
In this paper, attempt is made show effectiveness of transition based dependency parser for Amharic language. We constructed two network models for predicting arc-eager transition action and for predicting relationship types (dependency relations) using state of the machine learning techniques. The use of the second model if to minimize the number of classes from 2n+2 to n, where the number of dependency relations in the language. This helps each dependency relation to have more examples from Amharic treebank and increase the accuracy of the prediction. We train the models 1574 sentences and scores 91.54 and 81.4 unlabeled and labeled attachment score respectively. Results show the parser can be made more effective using larger Amharic treebank. Event though, Amharic dependency parser reported in this paper has performed well, is clear that there are pairs that not correctly attached. Improving the size of Amharic treebank and addition of morphological features to the treebank would probably results in improvement on the performance of the proposed system, which is
research direction
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
B. Gambäck: L. Asker. Experiences with developing language processing tools and corpora for amharic. 2010.
Baye Yimam Mekonnen Binyam Ephrem Seyoum, Yusuke Miyao. Universal dependencies for amharic. In LRCF, 2018.
G. A. Demeke and M. Getachew. manual annotation of amharic news items with part of speech tags and its challenges. 2016.
C.; Ballesteros M.; Ling W.; Matthews A.; Smith N.A. Dyer. Transition-based dependency parsing with stack long short-term memory.
NIVRE J.; HALL J.; NILSSON J.; CHANEV A.; ERYIGIT G.; KÜBLER S.; MARSI E. Malt-
parser: A iagguage-indeeeneer system for data-driven dependency parsing. 2007.
M. Gasser. A dependency grammar for amharic, 2010.
D. Chen; C. Manning. A fast and accurate dependency parser using neural networks. 2014.
Yuval marton; Nizar Habash: Owen Ranbow. Dependency parsing of modern standard arabic with lexical and inflectional features. In Association for Computational Linguistics, 2012.
J. Nivre. Two strategies for text parsing. 2005.
J. Nivre. "dependency parsing. 2010.
Reut Tsarfaty; Djamé Seddah; Sandra Kübler; Joakim Nivre. guages. 2013.
Parsing morphologically rich lan-
R. Rangra. Basic parsing techniques in natural language processing. 2015.


Reviews:
[{"review_id": "rye8rHBmsH", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper describes a dependency parser for Amharic text trained on the Yimam et al. 2018 treebank.\n\nThe proposed method is an unlabelled arc-eager transition-based dependency parser followed by a dependency label classifier. Both models are based on LSTM architectures. Unfortunately there is no clear description of these models, which makes it difficult to understand their structure.\n\nThe experimental section contains a questionable selection on the train/test split (which should have been done blindly before looking at the results) and lacks any baseline comparison with previous approaches. The authors argue that performing arc prediction and arc labeling in two separate stages is beneficial, but they don't compare with a method that perform both actions at the same time.\n\nThe motivation of the paper is also questionable: the introduction section describes some of the peculiarities of the Amharic language, such as being morphologically rich, but the proposed method seems completely generic and does not address or exploits any of these peculiarities. If the authors want to argue that Amharic would benefit from specialized parser architecture, the methods and results reported in this paper fail to provide evidence for this position.\n\nThe citation format is also broken.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "N/A", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "H1xIpYAstB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper claims to build a transition-based dependency parser for Amharic. To produce a parse tree, the paper takes a two-stage approach, first producing the structure of the tree, followed by annotating each individual arcs on the tree. The paper also explores attention and finds it useful for improving the performance.\n\nI am giving a score 1, because the paper lacks precision and the contribution is not significant.\n\nThe major problem of the paper is precision. There is a lack of formal descriptions of the parser, e.g., the representation of the input and output, what each individual block in the figures are doing, how the arc-eager transition parser works. The text description is certainly helpful, but without formal descriptions, it is impossible to reproduce the results in the paper. Besides, without setting up a precise language/notation, the paper is not easy to follow.\n\nIn terms of contribution, the paper begins with the motivation that Amharic has certain language characteristics that are absent in languages, such as English, where most research effort has been spent. However, the paper still uses a parser for English, and does not utilize the special features of Amharic. Generic architectural modification, such as adding attention, has been explored, but this has little contribution on its own and has little to do with Amharic.\n\nIn addition to the above, other minor weaknesses of the paper includes typos, poor experimental practices, missing citations and related work, and other wording issues.\n\nIn particular, choosing the split of the data set that achieves the best result, as quoted below, is concerning.\n\n\"We experimented on 60/40, 70/30 and 80/20 train-test splitting ratios ... Therefore, we select 70/30 train-test splitting ratio throughout the experiments.\"", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "H1ek-2mRKr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "* overview \n- This paper describes a model for transition-based dependency parser, and tested on the Amharic treebank. It proposes a modification that is different from the common practice of combining the transition with dependency relation, namely to predict the transitions first, then the dependency relations.\n    \n* strengthens\n - Unfortunately, I can't find any strength in this paper. It is more like a course project than a research paper for ICLR.\n\n* weaknesses\n- The main idea of first unlabeled parsing then predicting labels is hardly novel, and it generally won't help, since the predicted labels could be useful features for determining the transitions. There is also no experiment to compare the proposed model to normal parsing model with 2n+2 predictions to support the author's claim.\n- Generally, the architecture of the model is poorly explained. For example, it seems that the \"embeddings\" for the words are just one-hot index vectors (since the number of dimensions are the same as the vocabulary size), which is not what we mean by embeddings. It is also unclear what the dot product of the embeddings means. The attention layer is again not explained at all, e.g. what are the input and output.\n- The splitting of train/test set does not make sense, the selection should not be influenced by the results at all. There is no reason not to use the standard split in UD. \n- I don't see what is the challenge in the discussion section, since UAS is by definition lower than LAS.\n- There numerous typos and grammar mistakes, and the citation format is broken.\n- In many places, the quoted presumably Amharic words are not shown.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper builds a transition-based dependency parser for Amharic, first predicting transitions and then dependency labels. The model is poorly motivated, and poorly described. The experiments have serious problems with their train/test splits and lack of baseline. The reviewers all convincingly argue for reject. The authors have not responded. 
Decision: Reject