On the Interpretability of Deep Learning Models for Collaborative Argumentation Analysis in Classrooms (2024)

Abstract: Collaborative argumentation holds significant potential for enhancing students’ learning outcomes within classroom settings. Consequently, researchers have explored the application of artificial intelligence (AI) to automatically analyze argumentation in these contexts. Despite the remarkable performance of deep learning models in this task, their lack of interpretability poses a critical challenge, leading to teachers’ skepticism and limited utilization. To cultivate trust among teachers, this PhD thesis proposal aims to leverage explainable AI techniques to provide explanations for these deep learning models. Specifically, the study develops two deep learning models for automated analysis of argument moves (claim, evidence, and warrant) and specificity levels (low, medium, and high) within collaborative argumentation. To address the interpretability issue, four explainable AI methods are proposed: gradient sensitivity, gradient input, integrated gradient, and LIME. Computational experiments demonstrate the efficacy of these methods in elucidating model predictions by computing word contributions, with LIME delivering exceptional performance. Moreover, a quasi-experiment is designed to evaluate the impact of model explanations on user trust and knowledge, serving as a future study of this PhD proposal. By tackling the challenges of interpretability and trust, this PhD thesis proposal aims to contribute to fostering user trust in AI and facilitating the practical implementation of AI in educational contexts.

## introduction collaborative argumentation refers to a dialoguebased activity in which participants engage in constructing, critiquing, and reconciling arguments through social interactions @xcite . within classroom settings, empirical evidence consistently demonstrates that collaborative argumentation fosters critical thinking and knowledge construction by integrating learned facts and knowledge, reasoning, justifying, and negotiating @xcite . to fully harness its potential, teachers are encouraged to instruct students how to argue, facilitate students' engagement, and effectively manage collaborative argumentation @xcite . however, it has been observed that many teachers face challenges to master the necessary skills to effectively promote collaborative argumentation in their classrooms @xcite . to address this issue, some researchers propose recording and analyzing argumentative discussions utterance by utterance, employing an evaluation rubric to assess whether adjustments in teaching strategies and support interventions are needed for future classes @xcite . however, for teachers who are already burdened with daily responsibilities, conducting such laborious manual analyses is not feasible. to tackle this challenge, researchers have turned to the application of natural language processing (nlp) and artificial intelligence (ai) techniques to automate the analysis of classroom argumentation @xcite @xcite . initially, conventional machine learning techniques were employed to examine various aspects of teachers' discourse and students' engagement @xcite . subsequently, deep learning techniques were increasingly adopted to achieve more accurate analysis. for instance, @xcite utilized transformer-based neural networks to train models that automatically analyze teachers' ability to attend to students' ideas. despite these advancements, it has been observed that teachers are hesitant to trust the decisions made by such models @xcite . they express significant concerns regarding the lack of transparency and interpretability in these models, which undermines their trust @xcite . deep learning models often consist of complex structures with multiple layers interconnected by thousands or even millions of neurons, making them appear as "black boxes" that provide users with direct decisions without revealing the underlying process of prediction. the lack of understanding regarding the internal workings and individual decisions of these models likely leads to user distrust and underutilization of these tools, which can have a significant impact on the deployment of ai @xcite and teacher instruction in this particular case. to enhance user trust in ai-powered models and systems, researchers have proposed leveraging explainable ai (xai) to unravel the working mechanisms and individual decisions, providing explanations of ai @xcite . as a result, various interpreting methods have been developed @xcite . a systematic review conducted by @xcite demonstrates that explanations provided by xai effectively increase user trust and transparency in ai tools. desipte the significant progress, the interpretability issue of deep learning models for collaborative argumentation analysis in the classroom context remains largely unexplored. hence, this phd thesis proposal aims to investigate whether explainable ai methods can be effectively utilized to explain deep learning models for classroom collaborative argumentation analysis. specifically, we train two deep learning models on authentic transcripts of classroom collaborative argumentation to automatically analyze argumentative moves (i.e., claim, warrant, and evidence) and specificity levels (i.e., low, medium, and high). subsequently, we employ four interpreting methods -gradient sensitivity, gradient input, integrated gradient, and lime -to explain the model predictions by quantifying the contributions of input. the experimental results demonstrate that all four interpreting methods effectively explain the model predictions, with the lime method yielding the most competitive outcomes. furthermore, we design a quasi-experiment to evaluate the impact of explanations on user trust in and knowledge of the ai-powered collaborative argumentation model. we aim to contribute to addressing the interpretability challenge in the field of ai-supported classroom teaching, potentially fostering user trust in ai and facilitating the practical application of ai in teaching contexts. ## computational experiment prior to providing visualized explanations for users, we carried out a computational experiment to assess whether the obtained token contributions accurately represent their significance to the model prediction. in particular, we chose student utterances for which the argument move and specificity labels were correctly predicted by deep learning models. based on the decreasing order of token contributions computed by the four interpreting methods, we removed the most critical words in a step-wise manner until nine words were eliminated. if the token contributions truly signify their importance in the prediction of deep learning models, the removal of the most importance ones would result in a substantial change in prediction accuracy. taking into account that random deletion could also lead to a change in prediction accuracy, we conducted a random deletion experiment for comparison purposes. in our experiment, we separately selected 9,547 utterances for the bert-based argument move model and 8,417 utterances for the bert-based specificity model, all of which had a length greater than 10. as depicted in figure 2 , the removal of words from initially accurately predicted utterances based on their contributions results in a substantial decrease in prediction accuracy compared to the elimination of words at random. for example, for the bert-based argument move model, eliminating nine words according to contributions computed by lime and ig causes the prediction accuracy to decline from 1.0 to 0.44 and 0.59, respectively, while random deletion only leads to a drop in prediction accuracy to 0.80. similarly, for the bert-based specificity model, removing nine words based on contributions calculated by lime and ig results in a decrease in prediction accuracy from 1.0 to 0.38 and 0.63, respectively, whereas random deletion only causes the prediction accuracy to reduce to 0.79. the experimental results suggest that the four interpreting methods can explain argument move and specificity analysis by effectively identifying crucial words within argumentation, with the lime method demonstrating the most exceptional performance in model explanation. thus, we will use lime to provide model explanations in the subsequent user experiment. following the successful validation of the explanations, we designed an experiment aimed at evaluating the impact of these explanations on user trust 511 in and knowledge of the deep learning models for argument move and specificity. we will implement it in future practice as a critical empirical study of the phd thesis. ## conclusion recognizing the significance of collaborative argumentation in teaching and learning, this study employs bert (i.e., a widely adopted deep learning approach) and authentic discussion transcripts to develop two models for automated analysis of argument moves (i.e., claim, evidence, and warrant) and specificity levels (i.e., low, medium, and high) within collaborative argumentation. given that the "black box" nature of deep learning models may raise trust concerns among users, four explainable ai methods are proposed to unpack model analysis and provide explanations. these methods include gradient sensitivity, gradient input, integrated gradient, and lime. the computational experiments demonstrate the effectiveness of these methods in explaining model predictions by computing word contributions, with lime exhibiting the most exceptional performance. consequently, this study aims to apply the developed model and the lime method for collaborative argumentation analysis and explanation. a quasi-experiment is designed to evaluate the influence of model explanations on user trust and knowledge, representing a future extension of this phd proposal. by addressing the challenges of interpretability and trust, this phd thesis proposal contributes to the field of ai-supported classroom teaching, potentially fostering user trust in ai and facilitating the practical implementation of ai in educational contexts. this proposal also has several limitations that should be addressed before the formal implementation of the quasi-experiment. first, the study utilizes only one dataset, leaving uncertainty about the applicability of the explainable ai methods to models on other datasets of classroom collaborative argumentation. second, although the explanations for collaborative argumentation analysis are designed in a visual format, it is unclear whether this is the preferred format for teachers and how it might impact their perception of the explanations. therefore, further research should focus on evaluating the proposed method across multiple datasets and conducting a preliminary experiment to identify the optimal visualization of explanations. this will help avoid confounding the effects of explanations on users' overall trust.