Saffron-v1 (Experimental)

Saffron-v1 is a ~100.09M-parameter English language model with a custom architecture (RoPE, RMSNorm, SwiGLU, grouped-query attention, QK-normalization) and a custom byte-level BPE tokenizer. It is the first model from the Abhilash AI Research Lab, the research initiative of Abhilash Construction Company.

Status: Experimental / Preliminary. This is an early proof-of-life training run. It is not instruction-tuned, English-only, and can produce inaccurate or nonsensical text. Not for production use.

Training

Parameters	100.09M
Tokenizer	custom byte-level BPE, vocab 32000
Data (curriculum, streamed + capped)	tinystories -> wikipedia -> fineweb_edu
Token budget	~1.0B tokens
Best validation loss	3.4631 (perplexity 31.92)

Data sources: TinyStories, English Wikipedia (wikimedia/wikipedia), and FineWeb-Edu (HuggingFaceFW/fineweb-edu), concatenated simple -> complex.

Files

• saffron.pt — best checkpoint (raw PyTorch state_dict + config)
• tokenizer/vocab.json, tokenizer/merges.txt — the byte-level BPE tokenizer
• train_summary.json — training summary

Usage

Weights are a plain PyTorch checkpoint. Load them with the training/inference code at https://github.com/shishodiaabhilash/saffron-v1 :

git clone https://github.com/shishodiaabhilash/saffron-v1.git
cd saffron-v1
# place saffron.pt in results/ and the tokenizer in tokenizer/
python -m src.sample --config configs/studiolab.yaml --prompt "Once upon a time"

Limitations

Preliminary research artifact from a small-scale run. No safety tuning, no benchmark claims, English only. Outputs may be factually wrong.