Automated Grammar Error Correction for Urdu using Deep Learning
Authors: Muhammad Munawwar Anwar, Abdul Samad, Muhammad Arsalan Hussain, Muhammad Huzaifa Riaz


Reviewer paper summaries: The authors propose a method for grammar correction for the Urdu language on a subset of grammatical errors. They gathered a dataset composed of ~1200 sentences and formulate a method for synthetic data generation of grammatical errors. The authors train a MT5 on raw data only and also on a combination of synthetic data and real-world data which yields improvements.
Summary of strengths: + The main strength of the paper is their proposed training and testing dataset for Urdu, an underrepresented language in literature.
Summary of weaknesses: - The paper has limited novelty and technical contribution
- The dataset seems like the most important contribution of the paper. However, the dataset is too small which makes the results from Table 5 to be evident (smaller training corpus -> worse results)
- What about training only on synthetic data? This should have been explored.
- The paper also has no comparisons with other state-of-the-art methods / architectures.
- The approached problem is extremely limited; besides focusing on grammar correction for Urdu, an underrepresented language, the authors only address a subset of grammatical errors. It would have been of interest if the authors showed that through fixing grammatical error performance increases on some downstream task for Urdu.
- The state-of-the-art is outdated, the most recent study is from 2020
- Algorithm 1 is extremely simple, similarly Algorithm 2 is superficial – both should be removed
- mT5 is a rather old encoder-decoder model, newer alternatives (Flan-T5, mT0) should have been explored. Similarly, generative models could have been finetuned for the task at hand
Comments, suggestions and typos: - Introduction: missing reference (line 077)
- Information about precision / recall / GLEU are not necessary to be included since they are known in the literature (a reference for GLEU suffices)
- Descriptions about gradient clipping / gradient accumulation / mixed precision are not necessary, they can be described in a single paragraph with appropriate references alongside the “Experimental setup” section.
- Tables are unintelligible for non-speakers of Urdu (Table 3)
- Better formatting is needed for Tables (remove borders)
- Table 1 is not needed, could just be referenced.
- Table 5 should be transposed, with metrics on the first row and the methods (training data variations) on the rows.
- Line 237: “... format and the sentences had to by manually”
Overall assessment scores: 1