Assessing the efficacy of large language models in generating accurate teacher responses (2023)

Abstract: (Tack et al., 2023) organized the shared task hosted by the 18th Workshop on Innovative Use of NLP for Building Educational Applications on generation of teacher language in educational dialogues. Following the structure of the shared task, in this study, we attempt to assess the generative abilities of large language models in providing informative and helpful insights to students, thereby simulating the role of a knowledgeable teacher. To this end, we present an extensive evaluation of several benchmarking generative models, including GPT-4 (few-shot, in-context learning), fine-tuned GPT-2, and fine-tuned DialoGPT. Additionally, to optimize for pedagogical quality, we fine-tuned the Flan-T5 model using reinforcement learning. Our experimental findings on the Teacher-Student Chatroom Corpus subset indicate the efficacy of GPT-4 over other fine-tuned models, measured using BERTScore and DialogRPT. We hypothesize that several dataset characteristics, including sampling, representativeness, and dialog completeness, pose significant challenges to fine-tuning, thus contributing to the poor generalizability of the fine-tuned models. Finally, we note the need for these generative models to be evaluated with a metric that relies not only on dialog coherence and matched language modeling distribution but also on the model’s ability to showcase pedagogical skills.

## introduction the advent of powerful open-source generative language models such as gpt-2 @xcite , t5 @xcite , opt @xcite , bloom @xcite , flan-t5 @xcite or llama @xcite has led to significant developments in conversational agents, opening avenues for various applications in education @xcite . such ai-driven educational dialogues offer the potential for skill improvement and personalized learning experiences, with intelligent tutoring systems increasingly gaining traction @xcite . however, deploying ai-based teachers in the educational ecosystem demands the careful modeling and evaluation of these agents to ensure their capability to address critical pedagogical concerns. @xcite created the ai teacher test challenge which follows the recommendations from @xcite (pp. 67-72) stating that, if we want to put generative models into practice as ai teachers, it is imperative to determine whether they can (a) speak to students like a teacher, (b) understand students, and (c) help students improve their understanding. taking inspiration from the ai teacher test challenge which asks whether state-of-the-art generative models are good ai teachers, capable of replying to a student in an educational dialogue this paper seeks to investigate the applicability of reinforcement learning (rl) techniques in the generation of ai teacher responses within educational dialogues. the ai teacher test challenge emphasizes the need for a systematic evaluation of generative models to ensure that they can effectively communicate with students, comprehend their needs, and facilitate their academic improvement. can we guide the language generator with rl to help it focus on these pedagogical requirements? @xcite organized the shared task hosted by the 18th workshop on innovative use of nlp for building educational applications on generation of teacher language in educational dialogues. following the structure of the shared task, in this study, we aim to evaluate the potential of combining state-of-the-art generative language models with reinforcement learning algorithms to generate ai teacher responses in the context of real-world educational dialogues sourced from the teacher student chatroom corpus @xcite . the natural baselines for the task at hand are sota closed-source models such as gpt-4, and fine-tuned open-source pre-trained models such as gpt-2 @xcite . we will evaluate these natural baselines before evaluating fine-tuned pre-trained models using rl techniques, that optimize for pedagogical quality. by exploring the role of reinforcement learning in guiding the generation of ai teacher responses, we aim to advance the discourse on the utilization of conversational agents in educational settings and contribute innovative ideas to the ongoing shared task on the generation of teacher language in educational dialogues at the 18th workshop on innovative use of nlp for building educational applications. the rest of this paper is structured as follows. section 2 offers a comprehensive review of relevant literature in the areas of ai-driven educational dialogues and reinforcement learning-based language generation. section 3 discusses the analysis and processing of the dataset prior to conducting any language modeling tasks. in section 4, the proposed model and its methodology for generating ai teacher responses in educational interactions are introduced. section 5 evaluates the effects of our approach on the quality and relevance of the generated ai teacher responses and highlights key observations. finally, section 6 concludes the paper and explores potential directions for future research. ## related work a variety of related literature exists in the realm of conversational teaching between a student and a teacher. in this section, we review several notable works addressing aspects of teacher-student dialogues, foundation models, and conversational datasets, which have contributed to the progress and understanding of generative models in educational contexts. ## data the shared task for bea 2023 is based on the teacher-student chatroom corpus (tscc) @xcite . this corpus comprises data collected from 102 chatrooms where english as a second language (esl) teachers interact with students to work on language exercises and assess the students' english language proficiency. ## methods the primary objective of our study is to investigate the potential of using in-context learning, supervised fine-tuning, and reinforcement learning to generate ai teacher responses in educational dialogues. our proposed methods will be evaluated using the teacher student chatroom corpus (tscc) dataset. in this section, we provide an overview of the three main parts of our methodology: in-context learning using gpt-4, supervised fine-tuning with existing models such as gpt-2 and dialogpt, and supervised fine-tuning with reinforcement learning using the rl4lms library @xcite . ## results in this section, we present the results and discuss the performance of gpt-4, fine-tuned gpt-2, and fine-tuned dialogpt models on the tscc dataset. we analyze the strengths and weaknesses of each approach and provide insights into their potential applications and limitations in an educational context. ## discussion conversational agents have the potential to revolutionize the teaching landscape by addressing several challenges and enhancing the overall learning experience for both students and educators @xcite . however, developing conversational agents that can behave like human teachers requires addressing several challenges @xcite . data challenges. as noted in the subsections above, the generations from the gpt-4 model outperformed all the fine-tuned models, with and without reinforcement learning. to this end, we put forward the proposition that an array of dataset features plays a crucial role in posing significant challenges to the fine-tuning process of generative models. these features include several dataset characteristics, including sampling, representativeness, prompt and response lengths, and dialogue completeness-upon manual inspection, we identified several dialogues to be cut off-pose serious challenges in achieving superior performance with fine-tuning. furthermore, upon random inspection of the generations from the fine-tuned models, we identified that these models seem to have learned simple, generic, often inappropriate yet correct responses such as "thank you" and "okay." while more recent language models have been shown to have high few-shot performance, we believe that fine-tuned models could adapt better to provide domain-specific responses in comparison. to achieve this, we emphasize the need for extending the current dataset to include longer prompts with more context. it is important to acknowledge that these models might not be as effective as desired in their response generation due to these intricacies. the current efforts made by the research community to collect and build quality datasets encompassing enough information about the educational task to enable ai teacher generative models to fully generalize in any context is what we assess to be the main focus that the community should adopt (jarratt, 2023). ## conclusion in this paper, we explored the potential of using large pre-trained language models and reinforcement learning for generating ai teacher responses in an educational context. we first presented a few-shot approach using the gpt-4 model, which demonstrated promising results in generating contextually relevant and fluent responses, but with limitations in generating pedagogically sound responses consistently. we then fine-tuned gpt-2 and dialogpt on the tscc dataset and evaluated their performance using bertscore and dialo-grpt metrics. we also proposed an approach using rl to optimize directly for pedagogical values. we hypothesized that several dataset characteristics (e.g., dialog completeness, sampling) pose challenges to achieving superior performance with fine-tuning. to this end, we recommend the extension of the dataset to include longer prompts with extended context. finally, we also draw attention to the need for more domain-specific metrics (in both evaluation and reward-based training) in enabling the generation of accurate, context-aware, and factually correct teacher responses.