PEEP -Talk: A Situational Dialogue-based Chatbot for E nglish Education (2023)

Abstract: English is acknowledged worldwide as a mode of communication. However, due to the absence of realistic practicing scenarios, students learning English as a foreign language (EFL) typically have limited chances to converse and share feedback with others. In this paper, we propose PEEP-Talk, a real-world situational dialogue-based chatbot designed for English education. It also naturally switches to a new topic or situation in response to out-of-topic utterances, which are common among English beginners. Furthermore, PEEP-Talk provides feedback score on conversation and grammar error correction. We performed automatic and user evaluations to validate performance and education efficiency of our system. The results show that PEEP-Talk generates appropriate responses in various real-life situations while providing accurate feedback to learners. Moreover, we demonstrate a positive impact on English-speaking, grammar, and English learning anxiety, implying that PEEP-Talk can lower the barrier to learning natural conversation in effective ways.

## introduction in the era of globalization, english is used as a worldwide international language @xcite . a number of countries have incorporated the acquisition of foreign language communication skills into their lifelong learning objectives @xcite . @xcite have identified several areas, including education, tourism, and business, where direct communication with people is crucial, and conversation skill is considered as the most essential of the various language competencies. however, students learning english as a foreign language (efl) do not have sufficient opportunities to practice real-life english conversations @xcite . to address this issue, recent technologies such as smart speakers and conversational models have been applied in english education @xcite @xcite . in particular, chatbots have shown promising results in improving the communication skills and learning motivation of students @xcite . despite their potential, existing chatbot-based educational platforms face several challenges in providing effective language learning experiences. these chatbots employ hand-crafted and patternmatching rules, limiting their communication ability and responding appropriately to out-of-situation utterances @xcite . furthermore, as smart speakers and conversational ai models are not fully considered for educational purposes @xcite , they cannot cover various topics in real-life activities or provide educational feedback. to address these challenges, we propose peep-talk, a situational dialogue-based chatbot for english education. it consists of a conversation module, context detector (cd), and grammar error correction (gec) modules. the conversation module generates proper utterances considering the given situations with our proposed dataset, called situation-chat. it covers a variety of realworld situations. to address the previous chatbots' inability to interact with dialogue out of topic or situation, the cd module changes the situation when the conversation digresses from the current situation. peep-talk also provides feedback on learners' utterances with the situation similarity score, linguistic acceptability score, and grammar error correction. we quantitatively verified the performance of each module of peep-talk and conducted a user study to verify its effectiveness in english education in a real-world environment. to the best of our knowledge, there have been few attempts in nlp research to conduct a user study that verifies the performance and satisfaction of integrated modules. the comprehensive evaluation of peep-talk demonstrates the potential of our situational dialogue-based chatbot and feedback approach for effective english education. we deploy our methodology on a demo site. the code and dataset 1 are publicly available to facilitate further studies. ## chatbots for english education the field of language learning has been revolutionized by the emergence of chatbots. while previous computer-assisted language learning (call) systems have provided lessons and practice in writing, vocabulary, grammar, and other areas, they have limitations in that learners eventually need an actual human, such as a teacher or a colleague, to practice conversation @xcite . chatbots have the potential to bridge this gap in call systems, as they offer more natural conversation and feedback similar to that of human instructors @xcite . commercial chatbots for english education have become increasingly popular, including andy, mondly speak now, duolingo, and babbel. andy is a virtual tutor application that aids users in learning english, while mondly features language lessons in various situations with a virtual teacher. speak now uses ai-driven virtual avatars as instructors for learning english. duolingo offers gamified language lessons and has incorporated chatbots into its system, while babbel features chatbots as part of its teaching methodology. the features of these existing chatbots for english education are also shown in however, one of the main problems with current english education chatbots is that they are unable to provide a genuinely free and natural conversation experience. many chatbots rely on rule-based systems that are limited in generating diverse and spontaneous responses to user input @xcite . as a result, learners often feel frustrated and disconnected from the language they are trying to learn @xcite . additionally, these chatbots may struggle to understand idiomatic expressions, provide personalized feedback, and recognize situational contexts. without engaging in meaningful and contextually relevant conversations, learners may struggle to develop the communication skills necessary for reallife interactions in english-speaking environments. ## peep-talk we introduce peep-talk and discuss its motivation and unique features in §3.1. then, we present the situation-chat in §3.2, which contains diverse and situational expressions. the overall architecture of peep-talk, including its three modulesconversation, context detector (cd), and gec, is described in §3.3. finally, §3.4 covers peep-talk's deployment. ## experiments in this section, the effectiveness of the conversation module is verified through several automatic evaluation metrics and human evaluation. also, we analyze cd and gec modules in appendix d and appendix d.3 respectively. ## conclusion in this paper, we propose peep-talk, an english education chatbot that allows learners to practice real-life conversations and receive feedback. our research, pioneering in the realm of situational dialogue-based chatbots, was substantiated through rigorous qualitative and quantitative experiments. furthermore, through user evaluations and satisfaction surveys, we confirmed that our method enhances educational efficiency and reduces learning anxiety for efl learners in real-world english teaching environments. this research is pivotal as it introduces a novel approach to chatbot-based english education, enhancing learning efficiency and engagement. in the future, we will also integrate additional features, such as learner progress tracking and an administrative dashboard for personalized instruction.