mcq-deberta-v3-best-v2

Fine-tuned DeBERTa-v3-base for 5-option Multiple Choice Question answering.

⚡ This is the lightweight base variant (86M params, ~0.4s/sample).
For the high-accuracy large model, see 👉 Shitanshu06/mcq-deberta-v3-large (MAP@3 = 1.0000).

---

Model Details

Property	Value
Base Architecture	microsoft/deberta-v3-base
Parameters	86M (0.2B)
Task	5-option MCQ (Multiple Choice Question)
Evaluation Metric	MAP@3 (Mean Average Precision at 3)
MAP@3 Score	0.9420
Inference Speed	~0.4s / sample (CPU)
Training Epochs	3
Max Sequence Length	192 tokens
Optimizer	AdamW + Linear LR warmup
Project	IIT Madras BS in Data Science — DL & GenAI (T2-2026)
Author	Shitanshu Jayprakash Chaurasiya · Roll No. 24F2006167

---

Model Comparison

Model	Parameters	MAP@3	Speed	Use Case
mcq-deberta-v3-large ⭐	435M	1.0000 ✅	~1.2s	High-accuracy
mcq-deberta-v3-best-v2 ⚡	86M	0.9420	~0.4s	Fast inference

---

Usage

With Transformers (Multiple Choice)

import torch
from transformers import AutoTokenizer, AutoModelForMultipleChoice

model_id = "Shitanshu06/mcq-deberta-v3-best-v2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForMultipleChoice.from_pretrained(model_id)
model.eval()

question = "What is the primary function of mitochondria?"
options = [
    "Protein synthesis",
    "Energy production (ATP)",
    "DNA replication",
    "Cell division",
    "Lipid storage",
]
labels = ["A", "B", "C", "D", "E"]

encoding = tokenizer(
    [question] * 5,
    options,
    truncation=True,
    padding="max_length",
    max_length=192,
    return_tensors="pt",
)
inputs = {k: v.unsqueeze(0) for k, v in encoding.items()}

with torch.no_grad():
    logits = model(**inputs).logits  # (1, 5)

probs = torch.softmax(logits, dim=1)[0]
ranked = probs.argsort(descending=True)

print(f"Predicted: {labels[ranked[0]]}")
print(f"Top-3 MAP@3 order: {' → '.join(labels[i] for i in ranked[:3])}")

Quick Inference with MCQPredictor

# Clone the project repo first:
# git clone https://github.com/24f2006167/dl-genai-project.git

from src.inference import MCQPredictor

predictor = MCQPredictor("Shitanshu06/mcq-deberta-v3-best-v2")
result = predictor.predict(
    question="What is the capital of France?",
    option_a="Berlin",
    option_b="Madrid",
    option_c="Paris",
    option_d="Rome",
    option_e="Lisbon",
)
print(result["predicted"])    # → 'C'
print(result["map3_order"])   # → 'C → B → D'

---

Training Details

This model was fine-tuned using DebertaV2ForMultipleChoice on the Smart MCQ Solver Challenge dataset.

Training configuration:

• Architecture: DebertaV2ForMultipleChoice
• Encoding: [question] + [choice_i] for each of 5 options
• Loss: Cross-entropy over 5 logits
• Max length: 192 tokens
• Optimizer: AdamW (lr=2e-5, weight_decay=0.01)
• Scheduler: Linear warmup + decay
• Epochs: 3
• Metric: MAP@3 (Mean Average Precision at 3)

---

Live Demo

🚀 Try it interactively on the Hugging Face Space:
Shitanshu06/smart-mcq-solver

---

Project Repository

📂 Full training code, notebooks, and deployment:
github.com/24f2006167/dl-genai-project

---

License

Apache 2.0

---

Built by Shitanshu Jayprakash Chaurasiya as part of the IIT Madras BS Data Science programme — Deep Learning & GenAI project (T2-2026)