Adithyaadiga12/Shlok-GPT

ShlokGPT — a 50M-parameter Sanskrit GPT trained from scratch

ShlokGPT is a decoder-only transformer (nanoGPT-style) trained from scratch on 920 MB of Sanskrit text (86M tokens). It generates classical Sanskrit in Devanagari across multiple registers — epic, Vedic, Purāṇic, tantric-ritual, and dharmaśāstra.

Model details

Parameters	~49.8M
Architecture	Decoder-only Transformer (nanoGPT-style)
Layers / heads / embd	8 / 10 / 640
Context length	256 tokens
Vocabulary	16,000 (SentencePiece unigram, Devanagari)
Training tokens	~86M (train 81M / val 4.8M)
Final val loss	4.16
Trained on	Kaggle T4 GPU, float16, 40,000 iterations

Files

• final_ckpt.pt — model weights + config (PyTorch)
• model.py — model definition (needed to load)
• generate.py — inference script
• shlok.model, shlok.vocab — SentencePiece tokenizer

Usage

python generate.py \
  --checkpoint final_ckpt.pt \
  --tokenizer shlok.model \
  --prompt "धर्मक्षेत्रे कुरुक्षेत्रे समवेता युयुत्सवः" \
  --max_new_tokens 150 \
  --temperature 0.9 --top_k 50

What it's good at

• Continuing a Sanskrit prompt into coherent, grammatical classical Sanskrit
• Multiple registers: epic (Rāmāyaṇa/Mahābhārata), Vedic, Purāṇic, tantric ritual
• Correct morphology (dual forms, participles), real proper nouns, dialogue structure

Limitations

• Base language model — it continues text; it does not follow instructions or answer questions.
• Sanskrit/Devanagari only — it has never seen English; English input produces garbage.
• Not authoritative — it can generate plausible-but-fake verses. Do not rely on it to reproduce real scripture or meanings.
• Register mixing — some Buddhist/Pali content is present in the training data.

Training data & licensing

The corpus was assembled from public sources including GRETIL (CC BY-NC-SA 4.0), the Digital Corpus of Sanskrit, the Bhagavad Gita, and other public-domain texts. Because parts of the source corpus are non-commercial (CC BY-NC-SA), this model is released under CC BY-NC-SA 4.0: attribution required, non-commercial use, share-alike.

Citation

If you use this model, please credit the source corpora (GRETIL, DCS, et al.) and this repository.