Using Adaptive Empathetic Responses for Teaching E nglish (2024)

Abstract: Existing English-teaching chatbots rarely incorporate empathy explicitly in their feedback, but empathetic feedback could help keep students engaged and reduce learner anxiety. Toward this end, we propose the task of negative emotion detection via audio, for recognizing empathetic feedback opportunities in language learning. We then build the first spoken English-teaching chatbot with adaptive, empathetic feedback. This feedback is synthesized through automatic prompt optimization of ChatGPT and is evaluated with English learners. We demonstrate the effectiveness of our system through a preliminary user study.

## introduction teacher empathy has been shown to improve the learning experience, including increasing learner engagement and reducing anxiety @xcite @xcite . recently, @xcite suggests that students' perceived affective support (pas) from teachers has a positive correlation with l2 grit, defined as the passion and perseverance for second-language learning @xcite . pas generally corresponds to the perceived level of support for emotional needs. its definition includes caring, valuing responses, listening, and encouragement @xcite . we therefore expect empathy to correlate positively with pas. we aim to examine whether an empathetic, english-teaching system with high pas similarly boosts l2 grit. english-teaching systems have adopted affective components for various purposes @xcite . however, the systems that have introduced empathetic components into pedagogy are either situated in ubiquitous learning environments @xcite or are not naturalistic or seamless in their approaches to accounting for student affect @xcite ). an interactive system that effectively detects and alleviates esl learner anxiety without sensors (e.g. pulse rate monitors) or cameras has yet to be implemented. detecting negative emotion from a learner's audio is a promising way to offer empathetic feedback. however, off-the-shelf english speech emotion recognition models are often trained on data collected with native speakers of english @xcite . we hypothesize that english spoken by non-native speakers will have differences @xcite that challenge these off-the-shelf models. to address this, we develop a preliminary pipeline for this task using annotated audio data and incorporate it into a spoken empathetic chatbot system. our spoken chatbot detects negative emotions or prolonged pauses and then responds empathetically to encourage students. this negative-emotionresponsive design is inspired by an automated physics tutor that senses student frustration using sensors and cameras @xcite . we currently employ model-based and automatic approaches for sensing negative affect in user audio. the chatbot also provides grammar feedback. we utilize a grammar correction model for grammatical feedback and chatgpt with optimized prompting for empathetic feedback. positive preliminary user study results indicate that users perceive affective support from our system, paving the way for future large-scale experiments to study our system's effect on learner l2 grit. our main contributions are: 1) we release a dataset of mandarin-accented english speech with high-quality asr transcripts and negative emotion annotations, and 2) we propose the first sensor-free educational english chatbot that detects negative affect and intervenes by providing adaptive empa-thetic feedback foot_0 . ## user study fourteen native mandarin speakers were recruited from social media and the authors' connections. each participant conversed with the chatbot for at least 10 turns (a turn is one round of exchange between the chatbot and the user). a pre-survey for participant english proficiency and a post-survey for user experience were administered. in the presurvey, we obtain an approximate assessment of the participants' english proficiency including their standardized test scores, self-reported proficiency, and the frequency at which they speak english daily. after the participants interacted with our system, they were presented with a post-survey which includes a modified version of the teacher affective support scale (the last four items below) @xcite adapted for our context and general evaluations of conversation quality. our likert-scale post-survey includes: quality: how was the conversation quality? confidence: do you feel that you are more confident after conversing with the chatbot? useful: do you think the chatbot's grammar feedback is useful? encourage: the chatbot encourages me when i am having difficulties in the conversation. listen: the chatbot listens to me when i have something to say. care: my opinion matters to the chatbot. praise: the chatbot recognizes and appreciates when i am good at something. details for the surveys can be found in appendix h. example conversations between the participants and the bot can be found in appendix g. ## conclusion and future work in this work, we propose the negative emotion detection task in the context of english learning to capture learner frustration and anxiety. we also introduce the first english-teaching chatbot that provides adaptive, empathetic feedback to students using our negative affect detection pipeline. initial trials with end users demonstrate the potential of our system. for future work, we intend to scale up our user evaluations and verify our hypothesis that our system can effectively improve student l2 grit. for future work, apart from addressing participant feedback, we intend to expand our experiments to include more thorough comparisons between the different experimental conditions to establish more robust results. specifically, we want to determine whether our adaptive empathetic feedback improves l2 grit more than no empathetic feedback or fixed feedback upon multiple chatbot interactions. another interesting topic to examine more closely would be whether humans behave and react similarly when conversing with chatbots and real-life english teachers. we intend to include participants from an esl course in our next study.