## 1 Introduction
Most notably, they have failed to systematically integrate the full spectrum of Bloom's cognitive levels into the generation process, particularly struggling with the higher-order categories of analysis, evaluation, and creation. Additionally, previous work has not adequately addressed the specific challenges of IT education, where technicalaccuracy and currency of content are as important as cognitive alignment. Our preliminary survey of 30 IT educators confirms these gaps, revealing widespread dissatisfaction with existing tools and highlighting the critical challenges mentioned above that impede their adoption. To address the critical challenges of quality and effectiveness in automated assessment item generation within IT education, this research poses the core question:  
How does the Multi-Examiner system, by integrating knowledge graphs, domain-specific search tools, and large language models, improve the quality and effectiveness of automatically generated assessment items in IT education? This core question is examined through the following key evaluation dimensions or research sub-objectives:  
* Evaluating the ability of Multi-Examiner to generate contextually relevant and accurate distractors, thereby supporting meaningful and precise assessment in IT education. * Assessing the improvement in question diversity produced by Multi-Examiner across different cognitive levels of Bloom's taxonomy to ensure comprehensive evaluation of learners' knowledge and skills. * Measuring the effectiveness of Multi-Examiner's generation of higher-order thinking questions aimed at fostering critical thinking and deeper understanding in IT education. These dimensions guide the investigation to address the shortcomings of automated question generation systems in the context of IT education. To answer the core question, we propose Multi-Examiner, an innovative multi-agent system that integrates KGs, domain-specific search tools, and LLMs. This integration aims to enhance the contextual relevance, diversity, and cognitive depth of automatically generated IT MCQs. Our approach builds upon recent advancements in artificial intelligence and cognitive science, systematically integrating [PERSON]'s taxonomy principles into each component of the system design, while maintaining alignment with established educational principles and assessment frameworks. The system design was further informed by extensive consultation with IT educators, ensuring its practical value in real educational settings. The primary contributions of this study are fourfold. First, we develop a comprehensive KG and KB for IT education, meticulously categorized according to [PERSON]'s taxonomy, enhancing the system's capacity to generate contextually rich questions while maintaining pedagogical alignment. Second, we design and implement Multi-Examiner, a novel multi-agent system that synergistically combines KGs, domain-specific search tools, and LLMs to improve the quality, diversity, and cognitive depth of generated questions. Third, we introduce a multidimensional evaluation rubric that assesses both technical and pedagogical aspects of generated questions, facilitating continuous improvement of the QGS. Fourth, we conduct extensive empirical evaluation through rigorous testing with 30 experienced IT teachers, benchmarking the system's effectiveness against both GPT-4 and human-crafted questions. The subsequent sections of this paper are organized as follows: Section 2 provides a comprehensive review of related work in QGSs, with a focus on approaches utilizing KGs, LLMs, and educational taxonomies. Section 3 details the methodology, including the architecture of Multi-Examiner and the evaluation process. Section 4 presents the results of our comparative study, while Section 5 discusses the implications of our findings for both educational technology and assessment practices. Finally, Section 6 concludes the paper with a summary of contributions and directions for future research.