Exploring Methods for Generating Feedback Comments for Writing Learning (2021)

Abstract: The task of generating explanatory notes for language learners is known as feedback comment generation. Although various generation techniques are available, little is known about which methods are appropriate for this task. Nagata (2019) demonstrates the effectiveness of neural-retrieval-based methods in generating feedback comments for preposition use. Retrieval-based methods have limitations in that they can only output feedback comments existing in a given training data. Furthermore, feedback comments can be made on other grammatical and writing items than preposition use, which is still unaddressed. To shed light on these points, we investigate a wider range of methods for generating many feedback comments in this study. Our close analysis of the type of task leads us to investigate three different architectures for comment generation: (i) a neural-retrieval-based method as a baseline, (ii) a pointer-generator-based generation method as a neural seq2seq method, (iii) a retrieve-and-edit method, a hybrid of (i) and (ii). Intuitively, the pointer-generator should outperform neural-retrieval, and retrieve-and-edit should perform best. However, in our experiments, this expectation is completely overturned. We closely analyze the results to reveal the major causes of these counter-intuitive results and report on our findings from the experiments.

## introduction feedback comment generation is the task of generating explanatory notes for writing learning. an example of a feedback comment would be: feedback comment: since discuss is a transitive verb, about is not required. 2 this type of feedback can assist the writer in determining why their writing was wrong and how to fix it. while datasets @xcite @xcite for this task have become available, very little is known about the feedback comment generation methods. @xcite demonstrates deep neural network (dnn)-based retrieval methods are effective in generating feedback comments for preposition use, which is almost the only knowledge available. recently, a wide variety of dnn-based generation methods are available and they can improve feedback comment generation performance significantly. furthermore, nagata (2019) focused on feedback comments for preposition use. dnn-based retrieval methods might not perform well on more general feedback comments @xcite because they involve a wide range of writing items and rules. more sophisticated methods will likely perform better in such cases. given these contexts, we investigate feedback comment generation methods for preposition and general feedback comments in this study to demonstrate promising steps in this task. the first problem we must address is deciding which methods to investigate because several methods are available as mentioned above. following the findings, a retrieval-based method (which, will be referred to as retrieval-based, hereafter) is naturally chosen as a baseline method @xcite . however, retrieval-based is inflexible in that it can only output feedback comments existing in a given training data. dnn-based generation (i.e., sequenceto-sequence) methods demonstrate more flexible generation. we choose the pointer generator network @xcite as a simple dnn-based generation representative, which will be referred to as simple generation, hereafter. it is an encoder-decoder neural network with attention and copy mechanisms. it is preferable to have a copy mechanism because feedback comments often cite words from their target text. furthermore, a hybrid of these two methods that edit retrieved examples by the pointer generator network, which will be referred to as retrieve-and-edit, hereafter. retrieve-and-edit, which can benefit from retrieval and dnn language generation, is expected to perform better than generating feedback comments from scratch. to summarize, theoretically, performance order is expected to be retrieval we investigate the generation results to reveal the following four findings: edit generate more feedback comments that may mislead learners than retrieval-based because of their flexibility. based on these analyses, we reach the following conclusion. ## related work there has been little study on feedback comment generation. some researchers @xcite @xcite attempted to develop rule-based methods for diagnosing errors in line with grammatical error detection/correction. typically, rule-based methods parse input sentences and then apply rules to the resulting parse to diagnose errors. however, they encounter enormous difficulties in dealing with numerous errors and maintaining a broad set of rules. lai and chang (2019) proposed a template-based method that identifies error types and generates comments based on them. recently, datasets for research in feedback comment generation have become available. nagata (2019) proposed a feedback comment generation task with a dataset. nagata et al. (2020) extended the dataset to feedback comments in preposition use and in general. pilan et al. (2020) published a dataset that included feedback comments on linking word usage with error tags and learners' revisions. although datasets have become available, we know little about dnn-based methods for feedback comment generation, which have been proven to be effective in various natural language generation tasks @xcite @xcite . @xcite demonstrates that a neural-retrievalbased method performs well although they can only generate feedback comments existing in the training data. this implies that retrieve-and-edit-based methods, such as those proposed by @xcite , which are a natural extension of retrieval-based methods, are likely to be effective in this task. several variations on the retrieve-and-edit approach have been proposed. @xcite al. (2018); guu et al. (2018); cao et al. (2018); hossain et al. (2020) retrieve a similar instance using a superficial similarity of the entire input. in the feedback comment generation task, we must retrieve an instance of similar errors, and it is inappropriate to use superficial similarity because sentences with the same error are often superficially different. @xcite proposed a method for training a retriever to embed the input sentence in a task-specific manner. we use this model because we should retrieve instances based on the semantic similarity of the errors rather than the superficial similarity. ## methods to explore in this section, we describe the three feedback comment generation methods in detail. retrieval-based uses an architecture similar to that of sim-ple generation as a retriever. therefore, we explain simple generation, retrieval-based, and retrieve-and-edit in this order. in appendix a, we show the diagrams of these three methods. ## future direction based on the above findings, what direction should we take in the future in the task of feedback comment generation? first, simple generation is more effective when the type of error is limited, such as prep. the limited scope of feedback comment is a common situation in actual education. for example, students are taught a specific grammatical item in a certain lesson. however, when there is insufficient data for the type of error, as in the case of general, retrieve-and-edit can be effective if it is be improved. a better editor can be constructed by modifying the learning strategy and using external knowledge. if we can construct the proper editor, it is expected to output correctly for the 63 partially appropriate instances of retrieval-based (section 6.1). in addition, the latest architectures, such as bert, can be used for the encoder and decoder. our pilot study indicates that bert is effective as an encoder, but it needs further investigation. we leave this for future work. considering practical situations, it is not sufficient to simply consider the accuracy alone. the number of confusing outputs of simple gener-ation and retrieve-and-edit increases more than that of the retrieval-based model (section 6.3). when a learner receives the model's output as they are, the use of the generative model, which generates relatively many confusing comments, should be avoided. from another practical viewpoint, example-based methods, such as retrieval-based and retrieve-and-edit, have advantages that a simple generation does not have. for example, users can see the retrieved instances, which makes it easier to select the output. furthermore, when the retrieved instances are incorrect, the retrieved instances can be modified without retraining the model. these practical points of view suggest that each method should be used depending on the situation. ## conclusions in this study, we investigated three feedback comment generation methods. we showed counterintuitive results: (i) the performance of retrieve-and-edit was lower than the retrieval-based and (ii) the performance of the generation method varied depending on the task. furthermore, we analyzed the causes of these results. based on these analyses, the future directions are summarized as follows: (1) simple generation is effective in a setting with few variations of feedback comments such as prep, (2) retrieve-and-edit is expected to be promising for general feedback comment generation by suppressing over-editing and using external knowledge, (3) it will be necessary to develop a system that makes it easier for users to select the results, such as confidence estimation.