Personality-aware Student Simulation for Conversational Intelligent Tutoring Systems (2024)

Abstract: Intelligent Tutoring Systems (ITSs) can provide personalized and self-paced learning experience. The emergence of large language models (LLMs) further enables better human-machine interaction, and facilitates the development of conversational ITSs in various disciplines such as math and language learning. In dialogic teaching, recognizing and adapting to individual characteristics can significantly enhance student engagement and learning efficiency. However, characterizing and simulating student’s persona remain challenging in training and evaluating conversational ITSs. In this work, we propose a framework to construct profiles of different student groups by refining and integrating both cognitive and noncognitive aspects, and leverage LLMs for personality-aware student simulation in a language learning scenario. We further enhance the framework with multi-aspect validation, and conduct extensive analysis from both teacher and student perspectives. Our experimental results show that state-of-the-art LLMs can produce diverse student responses according to the given language ability and personality traits, and trigger teacher’s adaptive scaffolding strategies.

## introduction intelligent tutoring systems (itss) aim to offer individualized learning process, instant feedback, and dynamic knowledge tracing to learners @xcite @xcite . to align teaching activities with different characteristics and needs of students, itss leverage various techniques to generate learning contents, personalized instructional strategies and adaptive learning process @xcite @xcite . given the crucial role of dialogic teaching in stimulating and developing students' understanding, thinking, and reasoning, conversational itss could significantly im-* equal contribution. prove learning experience and outcomes @xcite . the recent emergence of large language models (llms) further reduces the reliance on domain-specific supervision from manual annotation (maclellan and koedinger, 2022), and can be adopted as tutoring agents for math @xcite , language learning @xcite , and social skill coaching @xcite . aside from delivering effective and fluent dialogic teaching, there is increased interest in exploring llms' potential for personalized education @xcite @xcite . in the real-world classroom, according to students' characteristics, human tutors adopt scaffolding strategies to improve their engagement and knowledge acquisition @xcite . among these characteristics, personality traits play a significant role in shaping stu-dents' learning style, motivation, and achievement @xcite . however, characterizing and simulating student's persona remain challenging when building and evaluating conversational itss. considering the complexity and diversity of language and persona, it requires a certain amount of real participants to construct the training data, and is difficult to scale up: the process of user recruitment, data collection, and annotation is labor-intensive and time-consuming, and student groups in pilot studies are often in small size and lack diversity. on the other hand, for quantitative evaluation, previous studies primarily focus on post-learning aspects, such as student feedback and learning outcomes @xcite , while pay less attention to personality-related dialogic analysis (e.g., scaffolding and engagement). in this work, we propose a personality-aware simulation and validation framework for conversational itss. to anchor a practical application, we conduct a case study on image description for language learning. as shown in figure 1 , for primary school students, image description and storytelling tasks are commonly used to assess and improve their language ability from word-and sentencelevel to discourse-level skills @xcite . to better reflect students' characteristics and language ability, we modulate the model's generation from both cognitive and noncognitive perspectives. more specifically, given that personality is one of the most influential noncognitive factors on language development @xcite , we refine and construct the five personality types for tutoring conversations (i.e., bf-tc) based on the big five theory (costa and @xcite , and integrate them into student simulation instructions. by modulating personality traits, one can collect diverse dialogue samples. to extensively evaluate of the simulation and reveal its pedagogical influence, we propose a multi-aspect validation, and conduct a quantitative analysis of the generated tutoring conversations at dialogue and utterance level, from student and teacher perspectives. our results on representative llms indicate that: (1) llms can follow instructions to simulate students with specified language abilities and personality traits, yet there remains a margin for improvement. (2) student simulation following our bf-tc scheme shows a high correlation with the vanilla big five theory. (3) llm-based tutoring systems are shown to adapt scaffolding strategies to fit different personality traits. our work highlights the importance of incorporating scalable, personalized simulations to better understand and enhance human-ai interactions in educational scenarios, and it paves a new way to the designing, developing, and evaluating conversational itss, ensuring a more engaging and effective learning environment tailored to diverse student needs. ## personality-aware student simulation & multi-aspect validation framework llms can perform as real users in task-oriented dialogues @xcite with natural communication and persona @xcite . in this work, we build a student simulator modulated by cognitive and noncognitive traits, and equip the framework with multi-aspect validation (figure 2 ). ## conclusion in this paper, we proposed a personality-aware simulation framework by integrating cognitive and noncognitive traits into tutoring conversations. we adapted the general big five theory for dialogic interaction, and enhanced the framework with multiaspect validation. our experiments and analyses under a language learning scenario showed that llms can be modulated by specifying personality traits to simulate different student groups and produce diverse responses, and scaffolding strategies would be adjusted upon student characteristics. our work emphasizes the need for scalable, personalized simulations to improve human-ai interactions, advancing the design and assessment of conversational tutoring systems for a more engaging and customized learning experience.