WORD SEQUENCE PREDICTION FOR AMHARIC LANGUAGE
Authors: Nuniyat Kifle
Ermias Abebe
Venue:  (2020)
Keywords: Word prediction
POS
Statistical approach

Abstract: Word prediction is guessing what word comes after, based on some current information, and it is the main
focus of this study. Even though Amharic is used by a large number of populations, no significant work is
done on the topic. In this study, Amharic word sequence prediction model is developed using Machine
learning. We used statistical methods using Hidden Markov Model by incorporating detailed parts of speech
tag and user profiling or adaptation. One of the needs for this research is to overcome the challenges on inflected languages. Word sequence prediction is a challenging task for inflected languages (Gustavii &Pettersson, 2003; Seyyed & Assi, 2005). These kinds of languages are morphologically rich and have enormous word forms, which is a word can
have different forms. As Amharic language is morphologically rich it shares the problem (Tessema,
2014).This problem makes word prediction system much more difficult and results poor performance.
Previous researches used dictionary approach with no consideration of context information. Due to this
reason, storing all forms in a dictionary won’t solve the problem as in English and other less inflected
languages. Therefore, we introduced two models; tags and words and linear interpolation that use parts of
speech tag information in addition to word n-grams in order to maximize the likelihood of syntactic
appropriateness of the suggestions. The statistics included in the systems varies from single word
frequencies to parts-of-speech tag n-grams. We described a combined statistical and lexical word prediction
system and developed Amharic language models of bigram and trigram for the training purpose. The overall
study followed Design Science Research Methodology (DSRM).


WORD SEQUENCE PREDICTION FOR AMHARIC LANGUAGE
September 2019
Abstract
The significance of computers, handheld devices and educational ap- plications are not deniable in the modern world of today. The interest of applying Al and Machine Learning algorithuss in this technologies grow- ing. Texts are entered these devices using word processing programs as well as other techniques. Word prediction is one of the techniques and as- sistive technology for people with disabilities. The main purpose of word prediction is to facilitate data entry and help dyslexic people reducing writing errors. Dyslexia is a learning problem some kids and few adults have. Dyslexia makes it lough read and spell. For developing countries such as Ethiopia this kind problems are neglected and the language spoken within the country are under resourced. Therefore this work has major contribution to those who have this type of disability in their language. Prediction can also help education by facilitating data entry. Huilding prediction model with better speed, grammatically correct sug- gestions and less search space is the main focus of the research. Amharic used by a large number of populations, however no significant work is done on the topic of word sequence prediction. In this study, Amharic word sequence prediction model developed with statistical methods ing Hidden Markov Model by incorporating detailed parts speech lag, some morphological features and user profiling or adaptation. Evaluation of the model is performed using developed prototype and keystroke ings (KSS) as a metrics. According to our experiment, prediction results using a bi-gram with morphological features and detailed Parts of Speech tag model has higher KSS and performed better compared those wothout Parts Speech lag. Therefore, statistical approach with detailed POS, morphological features like gender, number, and person with suggested root stem words using voice, tense, aspect, affixes statistical informa- tion and grammatical agrecment rules the language has quile good potential on word sequence Prediction for Amharic language.
1 Introduction
This research deals with designing word sequence prediction model in Amharic language. It is language that is spoken in eastern Africa (Wimsatt Wymn,

---

2011). One of the nceds for Amharic word soquence prediction for mobile use and other digital device is in order to facilitate data entry and communication in our language especially for those who has Dyslexia. Word sequence prodic- tion is a challenging task for inflected languages (Gustavii Pettersson, 2003; Seyyed Assi, 2005). These kinds of languages are morphologically rich and have enormous word forms, which is word can have different forms. As Amharic lan- guage is highly inflected language and morphologically rich it shares the prob- lem (Tessema, 2014). problem makes word prediction system much more difficult and results poor performance. Due to this reason, storing all forms in dictionary won't solve the problem in English and other less inflected languages. Hence considering machine learning algorithms that could help the predictor to suggest the next word with POS based prediction and the stated morphological features should be used.
1.1 background of study
Previous researches used dictionary approach with no consideration of context information. Hence storing all forms words in dictionary for inflected lan- guages such as Amharic language has been less effective. The main goal of this thesis is to implement Amharic word prediction model that works with better Prediction speed and with narrowed search space as much as possible. We in- troduced two models; tags and words and linear interpolation that parts of speech tag information in addition to word n n-grams in order to maximize the likelihood of syntactic appropriateness of the suggestions. believe the results found reflect this.
2
Methodology
The study followed Design Science Research Methodology (DSRM). Since DSRM includes approaches, tochniques, tools, algorithms and evaluation mechanisms the process, followed statistical approach with statistical language modeling using Machine learning algorithms and built Amharic prediction model based on information from Parts of Speech tagger. The statisties included in the sys- lems varies from single word frequencies to parts-of-speech lag n-grams. That means it included the statistics of Word frequencies, Word sequence frequencies, Parts-of-specch sequence frequencies and other important morphological infor- mation. Later on, the system was evaluated using Keystroke Savings. (Trnka Mecoy, 2008) Linux mint was used as the main Operating System during the frame work design. We used 31 tag sets that has been developed by linguis- ties professionals from differrnt domains, python programming language and its libraries for both the language model and the predictor module. Another Tool that was used is the SRILIM (The SRI language modeling toolkit) which was used to generate unigram, bigram and trigram count as an input for the language model. SRILIM toolkit that is used build and apply Statistical language modeling (Levy, 2015).
2

---

3
Conclusion
This study presents Amharic word sequence prediction model using the statis- lical approach. We described a combined statistical and lexical word prediction system for handling inflected langnages by making use of POS tags with mor- phological features to build the language model using Hidden Markov Model, TNT tagger. We developed Amharic language models bigram and trigram for the training purpose. We obtained 29We concluded that employing syntactic information in the form of parts-of-Specch (POS) and morphological features with u-grams promises more effective predictions. We also can conclude that data quantity, performance of POS tagger and data quality highly affects the keystroke savings. According to our evaluation, belter Keystroke saving (KSS) is achieved when using bi-gram model than the tri-gram models and higher n- grams. This is because in statistical modeling with higher n-grams requires huge amount data. We believe the results obtained were effective in reflecting bet- ter speed, correctness of suggestions (grammatical), and search space since these are the basic issues in word sequence prediction and in assistive technology.
3.1 Limitation
This research has limitation of data quantity and quality. We have tried our best to collect data sets and building such models needs much more data in statistical approach. The quality of POS Tagger has quite negative effect the prodictor module We have plan of extending this work using neural network algorithms like recurrent neural network and CNN with a better POS model. This algorithms overcome the data problem we face because they can work effectively with small data. We also would like to adopt this work for other local languages that has many speakers Ethiopia.
4 References
Gustavii, E., Pettersson, E. (2003). A Swedish Grammar for Word Prediction, (June). Levy, R. (2015). Working with n -grams in SRILM, (February), 1-6. Seyyed, M. G., Assi, M. (2005). Word Prediction in Running Text: Statistical Language Modeling for the Persian Language.
Tessema, T. (2014). Word Sequence Prediction for Amharic Language, (October).
Trnka, K., Mecoy, K. F. (2008). Evaluating Word Prediction: Framing Keystroke Savings. Computational Linguistics, (June),261-22 Wimsatt, A., Wynn, R. (2011). Amharic Language and Culture Manual.
3


Reviews:
[{"review_id": "rygH6_Mh5r", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "*What is this paper about?*\n\nThe authors propose a method to incorporate POS tags into a language model to improve its performance in Amharic language.\n\nShort review:\n\nThe authors tackle an interesting task which deserves more attention. Nonetheless, they do not fully describe their models or results with enough detail, so it is hard to evaluate this work.\n\n\nContributions:\n\nThis work tackles a relevant problem that seriously impacts speakers of low resource languages.\n\n*What strengths does this paper have?*\n\nIt tackles and interesting problem.\n\n*What weaknesses does this paper have?*\n\nThe authors do not present their models in enough details so that the reader fully understands it.\nThey also only gloss over the results, not presenting them in any concrete form, stating: “We believe the results obtained were effective in reflecting bet-ter speed, correctness of suggestions (grammatical), and search space since these are the basic issues in word sequence prediction and in assistive technology”. This referred results are not shown in the manuscript though.\n\n\n*Detailed comments:*\n\nThe paper does not use the official conference template. It is also very short, not going in details about the used techniques. Finally, references are not correctly formatted.\n\nSection 1.\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "HJxiqVt3tr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThis paper proposes to predict word sequences for Amharic language-- a language spoken in Eastern Africa. It proposes to use HMMs with POS tags and morphological features to perform this prediction task.\n\n\nThe paper is just 3 pages, contains 1 paragraph of methodology, and no experiments section. It is clearly a very early stage work and not in the scope of ICLR. This paper should have been desk-rejected as it needs more work before it is fit for publication. There is lot of work on word sequence prediction and HMMs are no longer the state-of-the-art. The authors should consider looking at RNN-based methods such as LSTMs for this task.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "N/A", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "BJeDrU0bKH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I would not like to sound offensive, but this paper is clearly below the standards of the conference, and outside any academic orthodoxy for the matter:\n- It is only 3 pages long including references, and does not even follow the conference template.\n- It has only 3 sections (\"introduction\", \"methodology\" and \"conclusions\") and 2 subsections (\"background of study\" and \"limitation\"), and all of them are only one paragraph long.\n- The work done is not adequately described, so it is not possible to say much about it, but it seems clear that there is no novelty nor sufficient rigor in it. The problem tackled is defined as \"word prediction\", which seems to be some form of language modeling. The proposed method combines HMMs with n-grams and morphological and POS features, which are all well established and should be considered more of a (nowadays outdated) baseline. There is no proper evaluation: the experimental settings are not described, and only one number is reported, with nothing to compare to.\n\nTo the authors: Please do not feel discouraged by my review. Your motivation (helping people with dyslexia in Ethiopia) is certainly laudable, but neither the work carried out nor its presentation meets the standards of our research community. I would suggest that you check some of the accepted papers in last year's conference to get a sense of what is expected. I assume that you are new to the field. Don't feel the need to publish your own research from the first day, and take your time to become familiar with the field and study the basic concepts. It takes time, but we all had to go through it :)", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "N/A", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a language model for Amharic using HMMs and incorporating POS tags. The paper is very short and lacks essential parts such as describing the exact model and the experimental design and results. The reviewers all rejected this paper, and there was no author rebuttal. This paper is clearly not appropriate for publication at ICLR. 
Decision: Reject