Sentence-level Feedback Generation for E nglish Language Learners: Does Data Augmentation Help? (2023)

Abstract: In this paper, we present strong baselines for the task of Feedback Comment Generation for Writing Learning. Given a sentence and an error span, the task is to generate a feedback comment explaining the error. Sentences and feedback comments are both in English. We experiment with LLMs and also create multiple pseudo datasets for the task, investigating how it affects the performance of our system. We present our results for the task along with extensive analysis of the generated comments with the aim of aiding future studies in feedback comment generation for English language learners.

## introduction grammatical error correction has been vastly studied recently in the nlp community @xcite , but it is not always sufficient to merely provide the learner with a correction; in many cases, explicit feedback can facilitate the learning process. language learners can revise improperly employed linguistic elements by reviewing feedback containing information on the error such as an explanation of why the usage is incorrect and suggestions on how to correct it. this will also help the user avoid making similar errors in the future @xcite . in this paper, we focus on preposition errors made by english language learners. some studies have shown that the majority of syntactic errors made by english language learners are prepositional errors of substitution, omission, and addition @xcite . prepositions are challenging for language learners to master since they are highly frequent; short, unstressed and perceptually weak; and can have several different senses which may not map onto their native languages @xcite @xcite . the task of feedback generation hasn't been explored much until recently when @xcite proposed the feedback comment generation task and a corpus @xcite and then organized the genchal 2022: fcg (feedback comment generation for writing learning) shared task @xcite . in this task, a system generates an explanation note, given a sentence and a span that indicates the error in the sentence. later, @xcite explored different baselines for this task, including a neuralretrieval-based method, a pointer-generator-based seq2seq model, and a retrieve-and-edit method. for preposition-related errors, they found the pointergenerator-based seq2seq model performs the best. in this paper, we describe our submission to gen-chal 2022: fcg @xcite . we use a simple encoder-decoder model to tackle the task and provide extensive analysis of the different aspects of the task. our contributions in this paper are as follows: ## results and analysis results of our experiments are available in we compared against the official shared task baseline system, which was an encoder-decoder with a copy mechanism based on a pointer generator network. ## discussion in this paper, we presented a simple baseline for sentence-level feedback generation for english language learners. we investigated the effect of using pseudo data for the task and provided an analysis of system outputs and metrics used. does data augmentation help? incorporating pseudo data gave us very slight bleu score improvements (0.14) over the t5 model with no pseudo data. but in human evaluations, we observe a 3.3% improvement in f1. our experiments show that pseudo data could potentially improve results if carefully created and incorporated. studies on similar tasks such as grammatical error correction have shown that there are many important factors when creating pseudo data such as choice of learner corpora, the method of generating the pseudo data, and error tendency of learners @xcite @xcite . future studies should further investigate similar important factors for this task and their effectiveness. is this a general evaluation of grammatical error feedback capability? even though our model outperforms the baseline for this task by a large margin, we think it is not a good indicator of current models' performance for the feedback comment generation task in general, due to the following limitations: 1) the topics covered in the train/dev/test sets are very limited and only related to smoking and part-time jobs. considering the topics and the fact that learners may have a limited vocabulary in the target language, the model is likely performing well because it has seen most of the errors during training, not because it has developed the capacity to recognize and comment on grammat-