Ajnyana-1

First Sundanese-only language model trained from scratch. Part of the Deflated Indonesian AI stack.

"Bringing ancient Sundanese wisdom into modern NLP."

Model Details

Architecture	nanoGPT (Karpathy-style)
Params	8.95M
Vocab	BPE 16K (Sundanese-specific)
Context	512 tokens
Layers	6
Dim	256
Heads	4

Training

Dataset	ajnyana-corpus
Tokens	~122M
Steps	10,000
Hardware	Kaggle T4 (~83 min)
Optimizer	AdamW, cosine LR 1e-3 → 1e-4, warmup 500
Final val loss	3.7538
Perplexity	52.20

Usage

import torch
from tokenizers import ByteLevelBPETokenizer

tokenizer = ByteLevelBPETokenizer("tokenizer/vocab.json", "tokenizer/merges.txt")
ckpt = torch.load("latest.pt", map_location="cpu", weights_only=False)
# See github.com/ripkiiii/ajnyana for full inference code

Limitations

• 9M params — syntactically valid Sundanese but semantically incoherent
• CC-100 source is noisy (SEO spam partially filtered)
• No instruction tuning

Links

• Corpus: ripkiiiii/ajnyana-corpus
• Code: github.com/ripkiiii/ajnyana
• Blog: deflated.xyz