S tu B ot: Learning by Teaching a Conversational Agent Through Machine Reading Comprehension (2022)

Abstract: This paper proposes StuBot, a text-based conversational agent that provides adaptive feedback for learning by teaching. StuBot first asks the users to teach the learning content by summarizing and explaining it in their own words. After the users inputted the explanation text for teaching, StuBot uses a machine reading comprehension (MRC) engine to provide adaptive feedback with further questions about the insufficient parts of the explanation text. We conducted a within-subject study to evaluate the effectiveness of adaptive feedback by StuBot. Both the quantitative and qualitative results showed that learning by teaching with adaptive feedback can improve learning performance, immersion, and overall experience.

## introduction learning by teaching is a teaching method that enables students to teach other students. many studies have revealed the effectiveness of learning by teaching from diverse perspectives, such as longterm learning @xcite , motivation and self-esteem @xcite , communication skills @xcite , and abilities to gather and structure information @xcite . some studies proposed situated and interactive learning with virtual agents using a learning by teaching approach, such as betty's brain @xcite , curiosity notebook @xcite , and simstudent @xcite . however, these studies mainly depended on structured interactions for the presentation (e.g., typing words in a form or clicking items). for example, the student had to use a structured concept map to express the causal relationships of a river ecosystem @xcite or to drag/drop math icons to teach formulas @xcite . in addition, choosing the following message or response among the given candidates was adopted to build a situation involving dialoguing with a virtual agent @xcite . although such structured interaction is convenient and suitable for well-defined and simple learning tasks, it limits the user's expression. few studies have addressed the earlier ideas of using conversational agents in learning by teaching to support users' free expression by allowing them to input texts @xcite . however, in these studies, giving feedback for learning by teaching was limited and rarely covered owing to the difficulties in understanding the user's input texts and in extracting what needs to be improved. in this paper, we propose stubot, a text-based conversational agent that promotes learning by teaching, as shown in figure 1 . stubot builds a virtual teacher-student situation. the user plays the role of a teacher by explaining a learning material to prepare stubot for an exam. based on the user's explanation texts, stubot continues to question insufficient parts of the explanation. to do this, stubot internally solves exercise problems through machine reading comprehension (mrc) that generates answer texts by analyzing the explanation. if the generated answer is incorrect, stubot asks the user a question about the key topic of the corresponding question. then, the user prepares another explanation for better teaching stubot. after several iterations, stubot finally takes an exam, and its exam results are shown to evaluate the user's learning and teaching. we conducted a within-subject study (@xmath0. the participants were asked to study the given learning materials by teaching stubot with and without adaptive feedback. the study results showed that the participants exhibited significantly greater learning performance with adaptive feedback from stubot. in addition, stubot increased the participants' immersion in learning by teaching. furthermore, the qualitative analysis results revealed the details about the participants' experiences with learning by teaching. finally, on the basis of the findings, we discuss the implications of designing a conversational agent for learning by teaching. ## related works many studies have proposed agent-based systems for learning, such as facilitating school administrative services @xcite and improving knowledge of a specific subject @xcite . furthermore, mathbot explains mathematical concepts to students and guides them in solving exercise problems @xcite . massistant helps users to trace the concepts they have learned and to build their own concept graphs @xcite . in particular, there have been virtual agents to motivate learners by situating them in a learning by teaching scenario. for example, betty's brain works in a scenario where the user prepares betty, a virtual agent as a student, for an exam to qualify for a science club membership @xcite . in betty's brain, the user explains a river ecosystem to betty by filling in the blanks of a conceptual map @xcite . similarly, in simstudent @xcite , the user helps a virtual agent pass a quiz. a virtual tutee system allows the user to build an intimate relationship with the conversation agent and teach it with responsibility @xcite . iwase et al. allowed the user to explain the theory of information technology to the virtual agent by selecting predefined dialogue sentences @xcite . entam teaches mathematics to virtual agents by dragging and dropping mathematical icons to explain a formula @xcite . such interactive systems for learning by teaching can be categorized into four subspaces according to two dimensions: (i) presentation and (ii) feedback. first, the interactive systems for learning by teaching can be distinguished by the existence of predefined structures for the presentation. many prior systems were based on structured interactions that allow the user to explain the learning material to an agent by selecting the responses or typing the values corresponding to specific attributes @xcite @xcite . in contrast, non-structural interactions are also possible with conversational agents @xcite . the system based on conversational agents enables users to deliver their lessons in their own words. this can take advantage of verbal explanations during learning. second, feedback types can also specify the systems for learning by teaching. most systems support adaptive feedback by highlighting the insufficient parts of the users' knowledge according to their previous presentations. such adaptive feedback to supplement previous understanding and explanations can improve the performance @xcite . prior systems mainly depended on structured presentations with adaptive feedback. the structured interaction can be convenient; however, allowing verbal expressions in the learner's own words can promote learning performance. furthermore, it is difficult to construct more complex learning tasks, such as summarizing historical events or paraphrasing literature articles. few studies have presented the initial idea and feasibility of using conversational agents for learning by teaching @xcite . however, the feedback was limited in these studies owing to the difficulties in processing the user's explanation texts. for example, providing feedback to users and correcting their previous teaching are not iterative (i.e., ending with the evaluation results rather than continuing the conversation) @xcite , or feedback messages were typed by a human teacher @xcite . this paper presents a new conversational agent that provides adaptive feedback to facilitate learning by teaching. as shown in figure 2 , stubot is designed to support the three steps of learning by teaching. first, in the preparation step, the learner is asked to prepare stubot for an exam. similar to the agent for counselor training @xcite , stubot uses a student persona to increase immersion in learning. a learning session begins with stubot's message, "i have a test this week. please teach me about the ooo." the learner then prepares a lesson that summarizes and explains the materials. second, the presentation step follows after the learner completes the preparation step. stubot allows the learner to explain using a free-text form similar to that used in conventional messengers. the learner can send a message to stubot by typing texts and pressing the enter button. the user can input the explanation texts in multiple subsequent messages, and the sent messages can be modified or deleted while the end button to finish the explanations is not pressed. finally, in the feedback step, stubot identifies insufficient parts by processing the user's explanation texts. it asks the user for further explanation about the insufficiently explained topics using feedback messages such as "can you explain ooo in another way?" the learner then prepares another explanation. stubot continues to provide feedback until the learner covers all the important topics or the given learning time is over. after some iterations, stubot finally takes an exam using all the previous explanations. then, the test results are delivered by showing stubot's answer to each problem and its actual answer, and the learner checks which parts are insufficient. ## implementation figure 3 shows three modules for analyzing user's explanation texts and generating feedback messages: answering, self-grading, and messaging. ## conclusion this paper presented stubot that asks the learner to teach a learning material and provides adaptive feedback to further explain the insufficient parts. to do this, stubot overcomes the limitations of existing conversational agents for learning by teaching by using advanced natural language processing. we conducted a within-subject study using stubot. the results reveal the effectiveness of stubot and provide practical implications for designing conversational agents for learning by teaching. this work can bring new opportunities for using mrc. in this study, mrc does not need to be perfect because stubot has a student persona. moreover, asking a question by stubot after the user's good explanation (i.e., error) was empirically acceptable because the user was a teacher who tries to explain better. we believe that our approach can extend application areas that mrc with practically technical limitations can apply.