More Insightful Feedback for Tutoring: Enhancing Generation Mechanisms and Automatic Evaluation (2024)

Abstract: Incorrect student answers can become valuable learning opportunities, provided that the student understands where they went wrong and why. To this end, rather than being given the correct answer, students should receive elaborated feedback on how to correct a mistake on their own. Highlighting the complex demands that the generation of such feedback places on a model’s input utilization abilities, we propose two extensions to the training pipeline. Firstly, we employ a KL regularization term between a standard and enriched input format to achieve more targeted input representations. Secondly, we add a preference optimization step to encourage student answer-adaptive feedback generation. The effectiveness of those extensions is underlined by a significant increase in model performance of 3.3 METEOR points. We go beyond traditional surface form-based metrics to assess two important dimensions of feedback quality, i.e., faithfulness and informativeness. Hereby, we are the first to propose an automatic metric measuring the degree to which feedback divulges the correct answer, that we call Informativeness Index I^2 . We verify in how far each metric captures feedback quality.

## introduction as technology continues to reshape the way we learn, an increasing number of people opt to learn new skills from the comfort of their own home. in this context, the traditional teacher role is often assumed by an automated tutoring system. ideally, this system's abilities should go beyond imparting knowledge and, like a real teacher, extend to the provision of response-specific personalized feedback that helps students uncover and fix errors in their current understanding. such feedback can manifest as an answer to any of the questions in figure 1 . each type serves a purpose and enriches @xcite . given a friend group taking a trip to the beach, a student is asked about their location and incorrectly answers that they stayed at home. the learning experience @xcite . should a student's answer suggest that they were not fully paying attention, a simple indication that an error has been made, i.e., verification feedback, might be sufficient to realign their focus. however, if they have major difficulties arriving at the correct answer or only miss small, less important details, one may wish to provide correction feedback instead. unfortunately, it is unclear whether all students fully understand their mistakes after receiving the correct answer. in fact, @xcite observed that instead, they tend to consider a task finished and do not reengage with the materials. in contrast, elaborated feedback eliminates possible answer options without disclosing the correct answer, forcing students to actively reevaluate their understanding. it helps students become more independent by identifying and correcting their own mistakes @xcite . the resulting sense of achievement has been shown to enhance motivation, making the learning process more rewarding and entertaining @xcite . yet, elaborated feedback is seldom applied in automated tutoring systems since its generation requires complex considerations pertaining to a knowledge source, the incorrect student answer, and the relation between them. that is, information related to the answer needs to be identified and then filtered based on whether the student seems to be aware of it or not. our contributions are as follows. we propose to employ a kullback-leibler (kl) regularization term between a standard and enriched input format to achieve more targeted input representations at inference time, promoting the identification of important parts of the input. the selective use of this presentation is encouraged by adding a preference optimization step that minimizes the entailment between the incorrect student answer and the generated feedback. finally, we assess in how far a range of traditional metrics can accurately reflect human judgments of feedback quality and suggest a new metric to capture the degree to which feedback divulges the correct answer. ## related work existing research on short-answer feedback generation focuses mainly on isolating and generating a single feedback type, i.e., correction feedback @xcite , explanation feedback @xcite , or hint feedback @xcite @xcite . some of those approaches use an external resource to retrieve relevant information prior to feedback formulation, such as machine-readable glossaries of textbooks (olney, 2021), domain ontologies @xcite or wikipedia entries @xcite . others draw the necessary information from a correct answer template alone @xcite . once all necessary information is collected, feedback is generated using either a template-based approach with fill-in slots @xcite , or more advanced language models. specifically, olney (2021) formulates feedback as the answer to a synthetic question asking, "what is the relationship between the student answer and the correct answer?" to generate its answer, they use the off-the-shelf long-form question-answering model eli5 @xcite . at the same time, other authors employ more general pre-trained encoderdecoder models, i.e., t5 @xcite and bart @xcite ), which they finetune on the target data. @xcite finetune t5 to predict a score or label and jointly explain it, while @xcite finetune both t5 and bart to produce a question hinting towards the reasoning required to arrive at the answer. as we intend to investigate feedback generation in the context of interactive reading comprehension exercises, the work most similar to ours is @xcite . they build a dialogue-based tutoring system that can adapt questions to the dialogue context, asses learner answers and generate diverse feedback types. to generate feedback, the system is provided with the reading passage, an individually passed excerpt of the passage, i.e. key sentences, the dialogue history including tutor question and incorrect student answer, as well as the correct answer. this system is implemented by finetuning distilgpt-2 @xcite on the direct dataset. we decided to construct additional data with more natural answers, including mistakes that students are likely to make in an environment where only the reading passage and no answer options are provided. for each question in the direct dataset, one annotator in the student role constructs such an answer, then another annotator in the tutor role constructs the corresponding feedback. both annotators are presented with the reading passage, the question, its correct answer, and the corresponding key sentences. five annotators with some level of english proficiency worked on the student role, while two native english-speaking annotators worked on the tutor role. the latter were also asked to periodically review randomly selected portions of the constructed data, including both incorrect answers and tutor feedback (constructed by the other worker). they ensured that the percentage of erroneous data items remained below 5%. this results in an additional 23,982 feedback turns. we call this new dataset direct-f foot_0 . to show differences and similarities to the direct dataset, we manually analyze 50 randomly sampled feedback turns each. as shown in ## problem definition given a reading passage p , a question about its contents q, and an incorrect student answer a s , the goal is to generate personalized feedback f . elaborated feedback, i.e., explanations and hints, is preferred over other feedback types. in this work, we do not draw from correct answer templates, assuming that the model can determine the correct answer independently and will learn more valuable representations along the way. ## experimental setup the combined direct and direct-f dataset is divided into train, validation, and test set, following the split of the race-m dataset. race-m assigns each passage with all its questions to precisely one split. during training, the model will see neither the questions nor passages that are later used for evaluation. this makes the test set especially hard to master. additionally, we save 5% of the original train set to be used for the optimization step. concrete data statistics can be found in appendix a. we limit the input length to 768 tokens. should the input exceed this limit, the end of the paragraph is truncated until the rest of the input fits the limit. the output is limited to 256 tokens. at inference time, we always pass the basic input without explicit key sentences and decode using beam search (@xmath0.0, which rewards longer generations. training parameters are described in appendix c. each model configuration was trained on five randomly chosen seed values and test set performance averaged across those. we report scores for sacrebleu @xcite @xcite , meteor @xcite , rouge-l @xcite and bertscore 4 @xcite . all metrics are computed using the hugging face evaluate library. for metrics that require an external tokenizer, i.e., rouge and meteor, we apply nltk's word_tokenize. the results are reported in ## results i compared to the t5 baseline, we observe significant performance improvements across all four traditional metrics for both proposed extensions. the absolute improvement varies across metrics between 1.8 for rouge to as much as 3.3 points for meteor. notably, dpo alone leads to an increase of 2.7 meteor points, while kl regularization 4 microsoft/deberta-xlarge-mnli with default baseline rescaling achieves a smaller but still significant (@xmath1.01) increase of 0.9 points. those results underline the effectiveness of our two proposed extensions. we will call the resulting full model rectify, i.e., to correct something or make something right. generation examples are shown in appendix b. since we have extended both the train and test set of the previously available data, the results above cannot be straightforwardly compared to related work. to make up for that, we retrain and evaluate our full model on the original direct dataset. @xcite used this data to train an endto-end tutoring system direct. while this system constitutes the only existing work on feedback generation for reading comprehension, feedback is only one of a number of possible turn types it can generate. this setup likely causes a performance bottleneck. our full model outperforms this system by 5.9 meteor points. b m r bs direct @xcite 5.1 24.5 22.3 22.3 rectify (ours) 8.9 30.4 25.3 26.2 ## results ii we now apply i 2 and f to our proposed model. i 2 @xmath2, while i 2 @xmath3. the number of instances falling in each category is given in square brackets. with each extension, the number of generations that lack important, helpful information decreases, and even those that still lack information get increasingly more helpful, as indicated by a consistent rise in i 2 @xmath4. at the same time, the overall faithfulness increases, but not as consistently. the drop in faithfulness when applying only the kl regularization term might be due to varying input formats that make it harder for the model to understand that the provided student answer holds wrong and not additional information. our proposed extensions are targeted towards providing more helpful information and they clearly fulfill this role. however, they are not targeted towards limiting this information accordingly. therefore, as the model enriches feedback, it starts divulging too much information, as indicated by a large drop in i 2 @xmath5. to investigate the true severity of this problem, we display the distributions of p e and p for our full model in figure 8 . about 27% of human feedback scores 0.8 or higher, i.e., an indicator of correction feedback. with about 34%, our model tends to produce significantly more correction feedback. still, it produces not only such feedback but hints of differing specificity as indicated by the distribution. ## conclusion and future work we proposed two extensions to the naive training pipeline to improve a feedback generation model's input utilization. to help identify essential parts in the source material, we proposed employing a kl regularization term between a basic and enriched input format. this step results in an improvement of 0.9 meteor points. then, in order to teach the model how to filter information, we added a preference optimization step. this step results in further gains of almost 2.4 meteor points. all in all, proving the effectiveness of our proposed extensions. using the newly introduced measure of informativeness i 2 , we identify a remaining weakness of the full model, i.e., a tendency to produce correction feedback. in the future, i 2 could be used in tandem with dpo to counteract this tendency.