zyoralabs/AQ-academic-ai-base

AQ-1B — Academic Quotient v1 (Base)

AQ (Academic Quotient) — India's Concept-First Academic AI. Raising the Academic Quotient of every student.

This is the pretrained base model. For the tutor (instruct) model — the one that answers student questions — see zyoralabs/AQ-academic-ai.

AQ-1B is a 1.26B-parameter foundation model built completely from scratch by Zyora Labs — proprietary architecture, own training code (pure PyTorch), own tokenizer, own data pipeline. No fine-tune of any existing model.

It is trained concept-first: the model learns the concepts of mathematics, physics, chemistry, biology, engineering, history, geography, civics and economics — from foundations to advanced — rather than curriculum checklists.

Highlights

• From scratch, end to end — architecture, tokenizer (32k byte-level BPE), training loop, and data pipeline all built in-house
• 20B tokens of knowledge-dense pretraining: encyclopedic text, real textbooks and course notes, scientific papers, and mathematical reasoning corpora
• Final quality anneal — the last 1.5B tokens use only the highest-quality sources (textbooks, course material, scientific papers, encyclopedic facts) with learning rate annealed to zero
• Tamil + Hindi inclusive — trained with native Tamil and Hindi text alongside English
• Progressive growth training — grown and continually trained through 75M → 300M → 1.26B parameter stages, each stage inheriting the previous stage's knowledge

Architecture (proprietary, from scratch)

Parameters	1.26B
Layers	48
Hidden size	1536
Attention heads	24 (grouped-query, 8 KV heads)
Feed-forward	SwiGLU, 4096
Positional encoding	Rotary (RoPE)
Normalization	RMSNorm
Context length	2048
Vocabulary	32,000 (byte-level BPE, English + Tamil + Hindi)
Embeddings	Tied

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("zyoralabs/AQ-academic-ai-base")
model = AutoModelForCausalLM.from_pretrained("zyoralabs/AQ-academic-ai-base", trust_remote_code=True)

ids = tok("Photosynthesis is the process", return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=60)
print(tok.decode(out[0]))

Intended use

AQ-1B is a base (pretrained) model — the foundation of the AQ educator stack (instruct tuning, retrieval grounding, and the AQ Playground sit on top of it). As a raw base model it predicts text continuations; it is not yet instruction-tuned.

Team

Name	Role	Affiliation
Vasanth	Chief AI Researcher	Zyora Labs
Adithi Sreedhar	Jr AI Engineer	AI & DS, Arunachala College of Engineering for Women

About

Built in India by Zyora Labs. AQ v1 is the first release of the Academic Quotient model family.