Qwen-BloomAware-Educational-MCQ-Generator

Overview

Qwen-BloomAware-Educational-MCQ-Generator is a specialized educational question-generation model fine-tuned to generate high-quality Multiple Choice Questions (MCQs) from educational content.

The model was developed as part of the Agentic AI Tutor project and focuses on producing pedagogically sound MCQs aligned with Bloom's Taxonomy. By leveraging educational metadata such as Bloom level, chunk type, concepts, and keywords, the model generates contextually relevant and educationally meaningful assessment questions.

---

Model Details

• Base Model: Qwen2.5-1.5B-Instruct
• Task: Educational MCQ Generation
• Training Method: Supervised Fine-Tuning (SFT) with Knowledge Distillation
• Teacher Model: Gemini
• Domain: Educational Content
• Language: English
• Final Selected Version: V8

---

Purpose

This model was developed to automate the generation of educational assessment questions from instructional materials.

Its primary objectives are:

• Generate high-quality MCQs from educational content.
• Produce questions across multiple Bloom's Taxonomy levels.
• Create plausible distractors.
• Preserve alignment with the source material.
• Support intelligent tutoring and adaptive learning systems.

The model serves as the MCQ generation engine within the Agentic AI Tutor platform.

---

Training Pipeline

The model was trained using a multi-stage pipeline specifically designed for educational question generation.

Stage 1: Document Processing

Educational documents were collected and processed from multiple formats:

• PDF
• DOCX
• PPTX
• TXT
• CSV

Text was cleaned, normalized, and prepared for downstream processing.

Stage 2: Semantic Chunking

Documents were divided into semantically coherent chunks using embedding-based segmentation techniques to preserve context and improve question quality.

Stage 3: Metadata Extraction

Each chunk was enriched with educational metadata, including:

• Bloom's Taxonomy level
• Chunk type
• Key concepts
• Keywords
• Question-generation suitability

Stage 4: Teacher-Based MCQ Generation

A high-capability teacher model (Gemini) generated MCQs using metadata-aware prompting strategies.

The teacher model received:

• Source chunk
• Bloom level
• Chunk type
• Concepts
• Keywords

and produced educationally aligned MCQs.

Stage 5: Automated Validation

Generated questions passed through a multi-stage validation pipeline including:

• Format validation
• Semantic relevance verification
• Distractor quality assessment
• Natural Language Inference (NLI) validation
• Duplicate detection
• LLM-based quality evaluation

Only validated MCQs were retained for training.

Stage 6: Dataset Construction

Validated MCQs were converted into an instruction-tuning dataset containing:

• Question
• Options
• Correct answer
• Explanation
• Source content
• Metadata annotations

Stage 7: Student Model Fine-Tuning

The Qwen2.5 base model was fine-tuned on the curated MCQ dataset using a knowledge-distillation approach.

The objective was to transfer the teacher model's educational question-generation capabilities into an efficient open-source model suitable for deployment.

Stage 8: Evaluation

Multiple model versions were trained and evaluated.

Version 8 (V8) achieved the strongest overall performance and was selected as the final production model.

---

Evaluation

The model was evaluated using both automated and qualitative metrics, including:

• Semantic Similarity
• Knowledge Retention
• Perplexity
• KL Divergence
• Composite MCQ Quality Score (CMQS)
• Bloom-Level Coverage
• Distractor Quality
• Question Relevance

Among all evaluated versions, V8 demonstrated the best balance between question quality, educational alignment, and generation consistency.

---

Intended Use

This model is intended for:

• Educational platforms
• Intelligent tutoring systems
• Question generation research
• Adaptive learning applications
• Assessment content creation

For best results, the input should include educational content and relevant metadata when available.

---

Related Repositories

Agentic AI Tutor

This model is deployed as part of the Agentic AI Tutor system:

https://github.com/Rehab-Hamdy/Agentic-AI-Tutor

Research & Training Repository

The complete experimentation workflow, dataset construction process, knowledge distillation experiments, hyperparameter tuning, and evaluation studies are available at:

https://github.com/Rehab-Hamdy/Quiz-Generation-KD-FT

This repository contains all experimental model versions. The current model corresponds to Version 8 (V8), which was selected as the final production model based on evaluation results.

---

Limitations

• The model is specialized for educational MCQ generation and is not intended for general-purpose text generation.
• Performance depends on the quality of the input content.
• Generated questions should be reviewed before use in high-stakes assessments.
• Educational quality may vary across domains not represented in the training data.

---

Citation

If you use this model in research or educational applications, please cite the associated Agentic AI Tutor project and the related repositories.