Distilling C hat GPT for Explainable Automated Student Answer Assessment (2023)

Abstract: Providing explainable and faithful feedback is crucial for automated student answer assessment. In this paper, we introduce a novel framework that explores using ChatGPT, a cutting-edge large language model, for the concurrent tasks of student answer scoring and rationale generation. We identify the appropriate instructions by prompting ChatGPT with different templates to collect the rationales, where inconsistent rationales are refined to align with marking standards. The refined ChatGPT outputs enable us to fine-tune a smaller language model that simultaneously assesses student answers and provides rationales. Extensive experiments on the benchmark dataset show that the proposed method improves the overall QWK score by 11% compared to ChatGPT. Furthermore, our thorough analysis and human evaluation demonstrate that the rationales generated by our proposed method are comparable to those of ChatGPT. Our approach provides a viable solution to achieve explainable automated assessment in education

## introduction student answer assessment is a critical component of the education process. prompt and insightful answer assessments can enhance students' learning experiences and support their academic growth @xcite . however, manually providing detailed feedback is timeconsuming, and differences in assessment criteria among various evaluators can result in inconsistencies in the grading process @xcite . various automated student answer assessment models have been proposed in recent years, mostly built on the pre-trained language models (plms) @xcite , making the assessment process more efficient and consistent. these approaches @xcite question: list and describe three processes used by cells to control the movement of substances across the cell membrane. student answer: active transport is when the movement is used with the need of energy. pasive transport is when the movement is used without energy. the sodium potassium pump is a type of movement that requires energy. ## conclusion in this paper, we proposed a framework called aera, which leverages the in-context learning and reasoning capabilities of chatgpt for rationale generation in student answer assessment. our experimental results suggest that although chatgpt is able to generate free-form rationales with natural language instructions, the example instructed prompt strategy achieves the best performance. we further demonstrate aera can effectively distil a smaller language model for efficient rationalization on automated student answer assessment tasks, without the need for additional human annotation on rationales. extensive experiments and human evaluation results have validated the efficacy of the refinement module, and our distilled language model can outperform the teacher model in grading while providing reliable rationales. our approach presents a cost-effective and efficient method for explainable automated student answer assessment.