Aryabhata: An exam-focused language model for JEE Math
Authors: name: Ritvik Rastogi; name: Sachin Dharashivkar; name: Sandeep Varma
Published: 2025-08-12 06:20:07

Abstract: We present Aryabhata 1.0, a compact 7B parameter math reasoning
model optimized for the Indian academic exam, the Joint Entrance Examination
(JEE). Despite rapid progress in large language models (LLMs), current models
often remain unsuitable for educational use. Aryabhata 1.0 is built by merging
strong open-weight reasoning models, followed by supervised fine-tuning (SFT)
with curriculum learning on verified chain-of-thought (CoT) traces curated
through best-of-n rejection sampling. To further boost performance, we apply
reinforcement learning with verifiable rewards (RLVR) using A2C objective with
group-relative advantage estimation alongwith novel exploration strategies such
as Adaptive Group Resizing and Temperature Scaling.
Evaluated on both in-distribution (JEE Main 2025) and out-of-distribution
(MATH, GSM8K) benchmarks, Aryabhata outperforms existing models in accuracy and
efficiency, while offering pedagogically useful step-by-step reasoning. We
release Aryabhata as a foundation model to advance exam-centric, open-source
small language models. This marks our first open release for community feedback
(https://huggingface.co/PhysicsWallahAI/Aryabhata-1.0{Aryabhata 1.0
on Hugging Face}); PW is actively training future models to further improve
learning outcomes for students.