E x ASAG : Explainable Framework for Automatic Short Answer Grading (2023)

Abstract: As in other NLP tasks, Automatic Short Answer Grading (ASAG) systems have evolved from using rule-based and interpretable machine learning models to utilizing deep learning architectures to boost accuracy. Since proper feedback is critical to student assessment, explainability will be crucial for deploying ASAG in real-world applications. This paper proposes a framework to generate explainable outcomes for assessing question-answer pairs of a Data Mining course in a binary manner. Our framework utilizes a fine-tuned Transformer-based classifier and an explainability module using SHAP or Integrated Gradients to generate language explanations for each prediction. We assess the outcome of our framework by calculating accuracy-based metrics for classification performance. Furthermore, we evaluate the quality of the explanations by measuring their agreement with human-annotated justifications using Intersection-Over-Union at a token level to derive a plausibility score. Despite the relatively limited sample, results show that our framework derives explanations that are, to some degree, aligned with domain-expert judgment. Furthermore, both explainability methods perform similarly in their agreement with human-annotated explanations. A natural progression of our work is to analyze the use of our explainable ASAG framework on a larger sample to determine the feasibility of implementing a pilot study in a real-world setting.

## introduction assessment is fundamental to any educational process as an evaluation system reflecting individual performance and a way to compare results across populations @xcite . two key elements to consider when designing an assessment are question type and grading method @xcite . while questions may come in various forms, such as multiple-choice questions, short answers, or essays, the grading method can be either manual grading performed by domain experts or automatic grading by computational methods @xcite . previous research has established that assessing free-text short answers is a process that, besides being time-consuming, may lead to inequalities due to the difficulties in applying consistent evaluation criteria across answers @xcite . data from several studies suggest that teachers dedicate approximately 25% to 30% of their time grading written examinations @xcite . moreover, manual grading requires concentration for long periods of time, which could lead to differences in grading for answers with similar quality, creating inequities in the assessment process and its outcome @xcite . in the literature, automatic short answer grading (asag) is defined as the task of assessing short natural language responses to objective questions using computational methods @xcite . asag techniques have evolved from traditional rule-based models to stateof-the-art systems utilizing deep learning-based natural language processing (nlp) models @xcite @xcite . researchers have been able to build supervised learning models based on assessment questions, answers provided by students, and the corresponding grades assigned by teachers @xcite . the objective is, therefore, to predict which label score a new question-answer pair should achieve. over the past five years, researchers have leveraged the power of novel deep learning architectures such as the transformer @xcite to improve accuracy for asag models @xcite . nevertheless, the performance improvement has come at the cost of models becoming less understandable for stakeholders, and their opaqueness has become an obstacle to their deployment in the educational domain @xcite . consequently, explainable artificial intelligence (xai) has emerged as a relevant research field aiming to develop methods that allow stakeholders to understand the outcome of deep learning-based systems @xcite . as such, several lines of evidence suggest that providing insights into models' inner workings might be helpful in building trust in these systems and detecting potential biases @xcite @xcite . a great deal of previous research into xai methods for explaining nlp models has focused on building reliable associations between the input text and output label and quantifying how much each element (e.g., word or token) contributes to the final prediction @xcite . such xai methods can usually be divided into feature importance-based explanations @xcite , perturbation-based explanations @xcite , explanations by simplification @xcite and language explanations @xcite . previous studies have indicated that rationales or language explanations are easier to understand and use since they are verbalized in human-comprehensible natural language @xcite . this study focuses on explaining binary text classification for student responses gathered from a data mining course exam. as such, the main objective is to generate a framework that predicts binary grades and simultaneously produces associated rationales in order to justify the predicted grade of a given student response. by doing so, we intend to enrich the insights given by previous research, by presenting a framework that demonstrates how recent progressions of deep learning architectures and xai can be combined in order to address the problem of asag. as such, we aim to set an example for how future research can incorporate xai in the educational domain. conclusively, our main contributions are as follows: ## related work large language models (llms) such as transformer models have been increasingly applied in the domain of asag @xcite . given a limited amount of examples, transformer models such as bert have proven their capability to achieve state-of-the-art performance within the field of asag @xcite . the ability to handle single short documents, such as questionanswer pairs, makes bert a suitable model for various downstream tasks @xcite . most previous research and implementations focus on the model's effectiveness using standard classification metrics such as f1 and accuracy, precision, and recall @xcite . however, there is a limited amount of research addressing why certain predictions are being made. as a consequence, a lack of trust and understanding of the model predictions remains an issue. thus, our work explores the use of explainability techniques as a tool for asag, in order to increase the understanding of the predictions being done. rationale extraction refers to a post-hoc explainability method for nlp models in which the goal is to create deep learning-based nlp solutions explainable by uncovering part of an input sequence that the prediction relies on the most @xcite . most previous research on rationale extraction has been carried out using an encoder-decoder architecture. in such a setting, the encoder works as a tagging model, where each word in the input sequence receives a binary tag indicating whether it is included in the rationale. the decoder then only accepts the input highlighted as a rationale and maps it to the target labels @xcite @xcite . previous studies have proposed a multi-task learning approach for rationale extraction utilizing two models and training them jointly to minimize a composite cost function @xcite @xcite . unfortunately, one of the main drawbacks of multi-task learning architectures for rationale extraction is that it is challenging to train the encoder and decoder jointly under instance-level supervision @xcite . pipelined models are a simplified version of the encoder-decoder architecture in which the encoder is first trained to extract the rationales. then the decoder is fit to perform prediction using only the rationale @xcite . it is important to note that no parameters are shared between the two models and that rationales extracted based on this approach have been learned in an unsupervised manner since the encoder is deterministic by nature. there is little consensus on what makes a good machine-generated rationale and how to evaluate a rationale for benchmarking. most researchers investigating rationale evaluation have utilized proxybased methods, where rationales are assessed based on automatic metrics that attempt to measure desirable properties @xcite . one of the most common methods for evaluating rationales is to measure how well they agree with explanations provided by human annotators @xcite . in the context of explainable nlp, this property is referred to as plausibility. as such, it is usually evaluated based on the token overlap between human annotations and machine-generated rationales. using such an approach, researchers have been able to derive token-level precision, recall, and f1 scores using intersection-over-union (iou) at token level @xcite @xcite . ## explainable autograding framework the explainable framework is illustrated in figure 1 , consisting of an encoder responsible for generating explanations and a decoder responsible of performing the binary classification. ## discussion when comparing the f1-score and recall of the shap method with the f1-score and recall of the ig method there seems to be little to no difference in their respective iaa with the human annotators. however, both of the methods seem to do well given the complexity of the data as well as the lim- given the scibert model and these accompanying explainability methods, it seems to be possible to generate representative explanations as well as explanations that could be valuable for a human annotator. given that the data set used is not only considerably smaller but also considerably more complex in terms of answer length than most data sets previously used in the task of asag, a slight decrease in classification performance is expected compared to previous research. furthermore, one implication of the classification results is that transformer models seem to require a very small amount of questionspecific data in order to substantially improve its performance in classification, even when given relatively complex data. however, such solutions may not replace human expertise. rather, using a combination of these models and the presented explainability methods, this performance can increase confidence in the given explanations and as a consequence, it could help aid and assist human experts in grading when data is very limited. ## conclusion and future work nlp tools hold immense potential for scoring freetext answers from students and augmenting teachers' evaluation capabilities in a scalable manner. transformer-based models can help identify patterns from students' responses and prioritize solutions that need further checking. however, their black-box nature becomes an obstacle when deploying these models in real-world educational applications. to bridge this knowledge gap, we introduce an explainable asag framework that produces competitive predictions along with humanunderstandable natural language explanations. our framework leverages llms capabilities combined with post-hoc explainability methods that do not require training, reducing the number of questionanswer pairs needed to achieve state-of-the-art re-sults. furthermore, the classification performance proves that llms can achieve competitive asag performance on complex questions with a low number of answers per question when given domainspecific training, indicating a low threshold for applying domain-specific asag. as a consequence, the resulting performance could give a certain degree of confidence when assisting teachers with valuable explanations. further work needs to be done to establish whether incorporating human-generated rationales during training can boost the model's predictive performance and the quality of its generated explanations @xcite . our future work aims to incorporate them using a multi-task learning approach and evaluate rationales beyond the plausibility dimension covered in the presented article. finally, we hope our framework and initial results can help promote research on explainability in asag systems.