KELE : A Multi-Agent Framework for Structured Socratic Teaching with Large Language Models (2025)

Abstract: Socratic teaching, known for its emphasis on heuristic questioning and deep thinking, has demonstrated significant advantages in promoting students’ cognitive development. However, traditional Socratic teaching places high demands on teachers’ expertise and real-time feedback capabilities, making it difficult to scale in large educational settings. Recent breakthroughs in large language models (LLMs) in natural language generation and dialogue comprehension offer the potential for automated Socratic teaching. In this paper, we propose Knowledge-Enlightened Learning Enhanced by LLMs (KELE), a novel multi-agent framework for structured Socratic teaching with LLMs. KELE constructs a structured Socratic teaching rule system (SocRule) and a “consultant–teacher” multi-agent collaborative teaching mechanism, in which two LLMs respectively take charge of teaching planning and execution, ensuring a logically coherent and hierarchically structured Socratic teaching process. We also construct SocratDataset, a structured Socratic teaching dataset covering 34 teaching strategies and over 42,000 dialogue turns, and train SocratTeachLLM, a specialized LLM for Socratic teaching tasks. Additionally, we build a comprehensive Socratic teaching quality evaluation system for LLMs, covering 9 dimensions from single-turn dialogue to multi-turn teaching processes. Experimental results show that SocratTeachLLM significantly outperforms GPT-4o, which has a much larger parameter size, across all Socratic teaching capabilities.

## introduction as a key mode of instruction, dialogue plays a crucial role in influencing learning outcomes @xcite . traditional knowledge-imparting teaching @xcite primarily focuses on delivering standard answers, placing students in a passive receiving role, as shown in figure 1 (a). in contrast, socratic teaching @xcite guides learners to actively think and construct knowledge systems through continuous heuristic questioning, as illustrated in figure 1 @xcite . existing research demonstrates that socratic dialogue can effectively promote learners' cognitive development @xcite . however, its heavy reliance on teachers' real-time feedback and personalized guidance makes the implementation process time-consuming and labor-intensive @xcite , which makes it difficult to be promoted in large-scale educational settings. the powerful capabilities of llms in natural language generation and real-time dialogue feedback @xcite @xcite offer a new technological pathway for automating socratic teaching by simulating instructor roles. current research on applying llms to socratic teaching remains in its early stages, primarily following two strategies: one involves prompt engineering to guide the model in generating responses with a socratic dialogue style @xcite ; the other fine-tunes models on specialized socratic teaching dialogue datasets to enhance its instructional guidance capabilities @xcite @xcite . however, current research generally focuses on superficial features of simulating socratic dialogue, such as heuristic questioning styles, while overlooking the importance of systematic teaching design and coherent dialogue planning. this often results in two prominent issues: 1) the absence of structured teaching processes and rule-based constraints can lead to arbitrary guidance, abrupt topic shifts, and disorganized feedback during interactions @xcite ; 2) the lack of mechanisms for summarization and reflection at the conclusion of teaching prevents learners from effectively consolidating and internalizing knowledge @xcite . these shortcomings not only weaken the potential of socratic teaching to stimulate deep thinking but also limit its controllability and practicality in real educational settings. therefore, incorporating structured teaching rules @xcite into llms is essential for enhancing the effectiveness and real-world applicability of socratic teaching. in this paper, we draw inspiration from rulebased socratic teaching @xcite and propose a knowledge-enlightened learning enhanced by llms (kele) framework. this framework implements knowledgeenlightened structured socratic teaching enhanced by llms, and consists of the following two components: 1) for dialogue scenarios employing llms to conduct socratic teaching, we have developed a structured socratic teaching rule system, socrule. socrule divides the socratic teaching process into five progressive stages. each stage is designed with specific dialogue strategies (34 in total) based on students' potential cognitive states and interactive behaviors during real learning processes, covering the complete teaching process from question raising to knowledge consolidation. 2) to effectively implement the socrule to realize structured socratic teaching, we innovatively propose a "consultant-teacher" multi-agent collaborative teaching mechanism, where two llm-based agents simulate the roles of a teaching consultant and a teacher in real-world educational scenarios. to enhance the socratic teaching capability of teacher agents within the "consultant-teacher" structure, we trained a socratic teacher large language model named socratteachllm. to build socratteachllm, we constructed a structured socratic teaching dataset called socratdataset. based on socrule, this dataset comprehensively covers 34 teaching strategies and includes over 42,000 rounds of teaching dialogues, specifically designed to strengthen the model's socratic teaching ability. we fine-tuned the glm4-9b (glm et al., 2024) on the socratdataset to obtain socratteachllm. this model serves as the core of the teacher agent, capable of generating thought-provoking questions and feedback, effectively guiding students through progressive questioning to facilitate deep thinking. additionally, to comprehensively evaluate the teaching quality of llms, we propose a socratic teaching quality evaluation system for llms. this system establishes a comprehensive evaluation framework covering nine dimensions, supporting multi-level analysis from single-turn dialogues to multi-turn teaching processes. to the best of our knowledge, this is the first systematic and generalizable evaluation framework for socratic teaching. experimental results demonstrate that the socrat-dataset significantly enhances the performance of llms in socratic teaching, and socratteachllm surpasses gpt-4o, which has several times larger parameter scale, in all teaching capability. the contributions of this paper are as follows: ## proposed kele framework the knowledge-enlightened learning enhanced by llms (kele) framework consists of two main components: the first is a structured socratic teaching rule system designed for llms, used to guide and regulate the teaching process; the second is a "consultant-teacher" multi-agent collaborative teaching mechanism, which effectively implements the rule system to realize structured socratic teaching. this section will elaborate on the design details and implementation methods of the above content. ## the socratic teacher large language model to further enhance the dialogue and guidance capabilities of the teacher agent within the "consultant-teacher" structure, we trained a llm with socratic teaching abilities under the guidance of the socrule: the socratic teacher large language model (socratteachllm). the development of socratteachllm involves three key steps: first, we constructed a highquality guided problem-solving dataset as the core question corpus. then, based on this dataset and socrule, we constructed a structured socratic teaching dialogue dataset called socratdataset. finally, we fine-tuned a sota llm glm4-9b on the socratdataset to obtain socratteachllm. the following sections will elaborate on the methods and implementation details of dataset construction and model training. ## conclusion this paper proposes the knowledge-enlightened learning enhanced by llms (kele) framework, offering a novel approach to applying llms in socratic teaching. we establish a structured socratic teaching rule system, socrule, and design a multiagent collaborative teaching mechanism composed of "consultant-teacher", which effectively implements structured socratic teaching and addressing the lack of systematic guidance in existing methods. to comprehensively evaluate the capability of llms in socratic teaching, we build a multi-level evaluation system covering single-round to multiround teaching processes, which is highly generalizable. additionally, we construct a structured socratic teaching dataset, socratdataset, based on which we train a socratic teacher llm, socrat-teachllm. experimental results show that socrat-teachllm excels across socratic teaching capabilities and significantly outperforms larger-scale models such as gpt-4o, further validating the effectiveness of the kele framework. in the future, we will continue to optimize this framework and explore its applications across different disciplines and diverse educational scenarios.