You need to agree to share your contact information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
meeny is free for personal, research, education & evaluation — and for commercial use under US$10,000/yr product revenue ("free until you make money"). Commercial use above that needs a paid license: US$49/year or US$99 lifetime, per product or domain. 60% off (code PPP60) for low/lower-middle-income countries. Scale (> US$1M revenue or > 100K MAU) = custom license. Buy → https://www.sprapp.com/buy · questions → sprappcom@gmail.com. By requesting access you agree to the Sprapp Model License v1.0 (SML-1.0).
Log in or Sign Up to review the conditions and access this model content.
meeny — a 6.2M ternary story model (BPB 0.519)
meeny is a TinyStories generator with ternary {-1, 0, +1} weights (≈1.58-bit, BitNet-style quantization-aware training) that runs in a browser tab, offline, no GPU. At 6.2M total parameters and a ~7 MB file it reaches bits-per-byte 0.519 on held-out TinyStories — substantially better than TinyStories-1M (0.707) and better than our own 999K eeny (0.625), with noticeably cleaner words and grammar.
Numbers (all measured)
| metric | meeny | eeny | TinyStories-1M |
|---|---|---|---|
| total params | 6,228,481 | 999,328 | ~3M |
| non-embedding params | 4,262,401 | — | ~1M |
| bits-per-byte (held-out val, lower=better) | 0.5186 | 0.625 | 0.707 |
| projection weights | ternary {-1,0,+1} | int8 | fp |
| speed (node WASM, single thread) | 343 tok/s (2.9 ms/tok) | 1949 | — |
| file size (KNM1 v3) | 6.99 MB | 1.76 MB | — |
Bits-per-byte normalizes cross-entropy by UTF-8 bytes — tokenizer-independent, so the comparison is fair.
What it is
- Decoder-only transformer: dim 320, 4 layers, GQA (5 heads / 1 kv), SwiGLU, RoPE, tied embeddings, 6144-token BPE tokenizer (the larger vocab removes the word-fragmentation seen in 4k-vocab tinies).
- Ternary QAT: projection weights {-1,0,+1} with per-tensor absmean scale (2-bit packed, 4 weights/byte), trained with a straight-through estimator — not post-hoc rounding. Embeddings f16, lm_head int8, norms fp.
- Knowledge distillation (KL T=2 + 0.5·CE) from a dim-576 / 10-layer full-precision teacher, 3B tokens on 8×RTX 5090.
- Runs on the Sprapp Rust→WASM engine (
gemv_ternarySIMD kernel) — the whole model generates in a browser tab, offline after first load.
Sample (greedy, in-engine)
Once upon a time, there was a little → girl named Lily. She loved to play outside in the sunshine
and run around with her friends. One day, she saw an orange ball on the ground ...
The dog ran to the → park. He saw a big tree with many leaves and branches. The
squirrel was very happy, but he did not know that there were some bees in the trees ...
Files
| file | what |
|---|---|
model_meeny.knm |
ternary weights, KNM1 v3 (6.99 MB) — for the Sprapp WASM engine |
meeny_v2.pt |
PyTorch checkpoint ({cfg, model}, fp latents pre-ternarize) |
tokenizer_meeny6k.json |
6144-vocab BPE tokenizer |
Limitations
- Domain: TinyStories only (simple children's stories). Not a general LM, no world knowledge.
- Ternary + tiny: occasional repetition or odd turns; greedy decode is most stable, temp ≤ 0.8 suggested.
Part of the Sprapp project — offline on-device tiny LMs in the browser (eeny / meeny / miny family). Trained on TinyStories (Eldan & Li, 2023).
License
Sprapp Model License v1.0 (SML-1.0) — free for personal, research, education, and evaluation.
Commercial use is free under US$10K/yr product revenue ("free until you make money");
above that: US$49/yr (or $99 lifetime) per product, 60% off in low/lower-middle-income countries (code PPP60).
Scale (> US$1M revenue or > 100K MAU) requires a custom license.
Buy: https://www.sprapp.com/buy · contact sprappcom@gmail.com. See LICENSE-MODELS.md.
Trained from scratch; TinyStories dataset attribution above.