Generating Inflectional Errors for Grammatical Error Correction in H indi (2020)

Abstract: Automated grammatical error correction has been explored as an important research problem within NLP, with the majority of the work being done on English and similar resource-rich languages. Grammar correction using neural networks is a data-heavy task, with the recent state of the art models requiring datasets with millions of annotated sentences for proper training. It is difficult to find such resources for Indic languages due to their relative lack of digitized content and complex morphology, compared to English. We address this problem by generating a large corpus of artificial inflectional errors for training GEC models. Moreover, to evaluate the performance of models trained on this dataset, we create a corpus of real Hindi errors extracted from Wikipedia edits. Analyzing this dataset with a modified version of the ERRANT error annotation toolkit, we find that inflectional errors are very common in this language. Finally, we produce the initial baseline results using state of the art methods developed for English.

## introduction grammatical error correction (gec) involves automatically correcting errors in written text, whether relating to orthography, syntax or fluency. today, most approaches for solving this problem highlight statistical and deep learning methods as opposed to rulebased methods. these methods treat gec as a translation task, from an ungram matical to a grammatically correct form of the same language @xcite . this re quires a considerable amount of supervised data in the form of 'edits', which are pairs of incor rect and correct sentences. researchers have re cently done remarkable work on english and a few other resourcerich languages and have released many datasets to evaluate state of the art meth ods. comparatively less attention has been given to low resource languages, and indic languages have been neglected in particular. systems like uttam @xcite and scmil @xcite have applied probabilistic approaches and deep learning, respectively, to the problem of spelling correction in indic languages. moreover, simple ngram based models @xcite have been used for "realword" error correction, which is a very sim ilar problem to gec. however, to our knowledge, no such work exists for true gec in this language. thus, we sum up our contributions in the following manner: ## related work the most common gec datasets come from correctionannotated language learner essays. the english learner corpora include those from shared tasks such as helping our own @xcite , @xcite and recently, bea2019 @xcite . similar learner corpora exist for the russian @xcite and the czech @xcite languages. however, the problem with such manually annotated corpora is that they are not readily available for low resource languages, and their creation will be resource and timeintensive. another popular method has been the deliberate injection of errors into grammatically correct sen tences, whether by a rulebased system or by strate gies like roundtrip translation @xcite . the former approach has been essential for languages with limited training data. this was the case for english early on @xcite , and is still the case for low resource languages such as indonesian @xcite . provided that the artificial errors closely resemble realworld mistakes, this method can be applied to obtain large volumes of training data reliably. a third approach involves mining edits from websites, such as language learner websites @xcite or from websites with public revision histories like wikipedia foot_0 @xcite or github @xcite . while this has the potential to yield natural datasets of considerable size, there are several issues with edits obtained by this method, as not all corrections made in the text are of a grammatical nature; and many simply add more information or are semantic improvements to the text. as the edits lack any hu man curation, this method results in a more noisy corpus. ## hindi grammar hindi is a fusional language that expresses gram matical features like case, gender, number, tense, etc. via morphological changes. in particular, all verbs and some adjectives are inflected to agree with the number and gender of the associated noun @xcite . the same is the case for gen itive pronouns, genitive postpositions, and ordi nals. additionally, the verb inflects for the person and the adjective declines for the case of the noun. with a few exceptions, these changes are indicated by vowel endings to the right of the lexical base, as shown with examples in ## error extraction from hindi wikipedia the wikiedits 2.0 foot_1 (grundkiewicz and junczys dowmunt, 2014) software uses wikipedia revision histories to extract a parallel corpus of errors. we modify this tool for hindi and extract edits from a wikipedia revision dump dated october 1, 2020 to create our dataset, which we term as hiwiked. for filtering the edits, we constrain extracted sentence length to between 6 and 27 tokens and consider only substitution operations with a tokenbased levenstein edit distance of less than 0.3. addition ally, we discard edits containing only a difference in punctuation or numbers and corrections involv ing extremely rare tokens or html markups. ed its relating to vandalism are also discarded. ## error analysis the error anotation toolkit (errant foot_2 ) @xcite is a tool that uses morphological and dependency information to analyze, merge and categorize errors using a rulebased system. initially created for english, it has since been extended to german @xcite . we use a similar method to extend the toolkit to hindi and use it to classify the errors in hiwiked (see although the classification criteria consider many exceptional cases, the basic reasoning used by us is as follows: ## artificial error generation since inflectional errors form an easy to identify and common class of hindi errors, we choose them to generate a synthetic dataset using the following process. we first extract sentences from the hindi wikipedia revision dated june 1, 2020 (using wikiextractor foot_3 ), assuming that the recent versions are mostly grammatically correct. we tokenize these sentences and pos tag them using the hindi pos tagger @xcite . we change the inflectional ending for all words of the verb, adp, adv and pron categories to a dif ferent random ending from the inflection table for that pos, taking care that exceptional cases are ad equately handled (for examples, refer ## experiments and evaluation in our experiments, we first test the feasibility of the system using a basic transformer architecture @xcite implemented using the ten sor2tensor foot_4 library. for this we use the trans former_base setting as a baseline and train the model for 5k epochs. we then evaluate slightly modified versions of two state of the art models relating to english gec. first, we train the multi layer convolutional encoderdecoder model (chol lampatt and ng, 2018) foot_5 for 5 epochs using the de for model evaluation, we use the gleu metric @xcite as well as the f 0.5 metric calculated using the maxmatch(m 2 ) scorer foot_7 @xcite . the systems were trained on the synthetic dataset and then eval uated on the hiwiked corpus, and the results are presented in ## discussion and future work motivated by the lack of work in gec for indic lan guages, we present two novel error corpora in the hindi language (as shown in in addition, some grammatical errors in hi wiked were not inflectional (such as adj:form) and thus not represented in the synthetic dataset. on manual observation of the dataset, we also find that some edits are identifiably incorrect or simply denote stylistic differences and are out of the scope of gec. thus, it may be fruitful to filter and an notate the dataset manually. including other error