TMU Feedback Comment Generation System Using Pretrained Sequence-to-Sequence Language Models (2023)

Abstract: In this paper, we introduce our Tokyo Metropolitan University Feedback Comment Generation system submitted to the feedback comment generation task for INLG 2023 Generation Challenge. In this task, a source sentence and offset range of preposition uses are given as the input. Then, a system generates hints or explanatory notes about preposition uses as the output. To tackle this generation task, we finetuned pretrained sequence-to-sequence language models. The models using BART and T5 showed significant improvement in BLEU score, demonstrating the effectiveness of the pretrained sequence-to-sequence language models in this task. We found that using part-of-speech tag information as an auxiliary input improves the generation quality of feedback comments. Furthermore, we adopt a simple postprocessing method that can enhance the reliability of the generation. As a result, our system achieved the F1 score of 47.4 points in BLEU-based evaluation and 60.9 points in manual evaluation, which ranked second and third on the leaderboard.

## introduction this paper describes our submission to the feedback comment generation task for inlg 2023 generation challenge @xcite . feedback comment generation is a task of automatically generating hints or explanatory notes about errors for the purpose of helping the language learner improve their writing skills @xcite . in this task, the target of the feedback comment generation is limited to preposition uses, such as missing prepositions, to-infinitives, and deverbal prepositions. in the previous study @xcite , pointer generator network @xcite was used as a sequence-to-sequence method and found 1 our source code is available at https://github.com/ noirued/t5_fcg.git input source sentence: i can not agree you in this case. offeset ranges: 9:18 ## our method we frame the feedback comment generation task as a sequence-to-sequence generation task. we finetuned the pretrained sequence-to-sequence language models with the official distributed datasets. since it is difficult for the models to learn the meaning of the offset ranges, instead of using offset as it is, we inserted the special tokens [boe] and [eoe] in the position of offset ranges. figure 2 shows the overview of our proposed method. ## conclusion in this paper, we described our submission to the feedback comment generation task for inlg 2023 generation challenge. the result of the experiments showed that using pretrained sequence-tosequence language models is effective in the feedback comment generation for preposition uses. furthermore, we found that using pos tags as an auxiliary input improves the generation quality, and confirmed that our postprocessing method enhances the quality of the feedback comments by editing unreliable feedback comments into reliable feedback comments. future work will explore additional postprocessing methods that can better identify and appropriately edit unreliable feedback comments.