Personalized Cloze Test Generation with Large Language Models: Streamlining MCQ Development and Enhancing Adaptive Learning (2024)

Abstract: Cloze multiple-choice questions (MCQs) are essential for assessing comprehension in educational settings, but manually designing effective distractors is time-consuming. Addressing this, recent research has automated distractor generation, yet such methods often neglect to adjust the difficulty level to the learner’s abilities, resulting in non-personalized assessments. This study introduces the Personalized Cloze Test Generation (PCGL) Framework, utilizing Large Language Models (LLMs) to generate cloze tests tailored to individual proficiency levels. Our PCGL Framework simplifies test creation by generating both question stems and distractors from a single input word and adjusts the difficulty to match the learner’s proficiency. The framework significantly reduces the effort in creating tests and enhances personalized learning by dynamically adjusting to the needs of each learner.

## introduction cloze multiple-choice questions are a prevalent form of assessment in educational settings. as depicted in figure 1 , a typical cloze test consists of a sentence with a blank and four answer choices: one correct answer and three distractors. testtakers are required to select the correct answer to fill in the blank. while high-quality distractors are crucial for accurately assessing students' comprehension levels, manually designing such distractors can be timeconsuming and labor-intensive. consequently, recent years have seen a surge in research focused on automating the task of distractor generation for cloze tests @xcite @xcite . despite the advancements in automated distractor generation, current methods produce nonpersonalized cloze tests that do not adjust to the difficulty based on a learner's abilities, overlooking the nuances of personalized learning as mentioned in @xcite . moreover, existing approaches typically require both a question stem and an answer as inputs. however, limited research has been conducted on generating a cloze test starting solely from a given answer, which includes creating both the corresponding question stem and distractors, as illustrated in figure 2 . this study addresses these gaps by introducing the personalized cloze test generation (pcgl) framework. using llms, the pcgl framework generates both the question stem and distractors from a single input answer, tailoring mcqs to match the user's difficulty level. the contributions of this study are as follows: figure 2 : this study aims to generate a cloze test that includes both the corresponding question stem and appropriate distractors for a given answer. recent methods for generating distractor options in cloze tests can be categorized into two main types: candidate generation and ranking (cgr) framework @xcite , and the generative text2text framework @xcite . in the cgr framework, cdgp (chiang et al., 2024) is considered state-of-the-art. it employs a candidate selection generator (csg) to create multiple candidate distractors and a distractor selector (ds) to choose the three most suitable words as distractors, based on lexical and contextual relevance. conversely, the text2text generation architecture, as described by @xcite , approaches distractor generation as a text2text task, where the question stem is concatenated with the answer before inputting into a generative language model (e.g., t5 or gpt) to train the model to produce a set of distractors. despite their advances, the cgr and text2text methods face significant limitations: they cannot adjust distractor difficulty levels and require a complete question stem with an answer. these constraints limit the adaptability of assessments and complicate the dg process. our study aims to address these shortcomings. ## methodology this study introduces a personalized cloze test generation framework, termed the pcgl framework, which leverages llms for generating mcqs tailored to the difficulty experienced by individual users. ## conclusion our research demonstrates that fine-tuning two pre-trained models and enabling their cooperation can generate a complete cloze task from a single word while also allowing for the adjustment of the task's difficulty level. although there remains room for improvement in fine-tuning the difficulty adjustments, the quality of the generated tasks already surpasses recent studies on cloze distractors.