eeny — a coherent TinyStories generator in 999,328 parameters
eeny is a tiny language model that writes short children's stories — and runs in a browser tab, offline, with no GPU. At 999,328 total parameters it is, to our knowledge, among the smallest coherent story models, and it beats TinyStories-1M (a model ~3× larger by non-embedding params) on a fair, tokenizer-independent metric.
Numbers (all measured)
| metric | eeny | TinyStories-1M |
|---|---|---|
| total params | 999,328 | ~3M (1M non-embedding) |
| bits-per-byte (held-out TinyStories val, lower=better) | 0.625 | 0.707 |
| in-browser speed (WASM, single thread, no GPU) | 417 tok/s (2.4 ms/tok) | — |
| file size (int8 KNM) | 1.76 MB | — |
Bits-per-byte is tokenizer-independent (normalize cross-entropy by UTF-8 bytes), so the comparison is fair across different tokenizers.
What it is
- Architecture: small decoder-only transformer (dim 88, 7 layers, GQA, RoPE, SwiGLU, tied embeddings), 4096-token BPE tokenizer trained on TinyStories.
- Trained by knowledge distillation from a small full-precision teacher on the TinyStories dataset, then exported to int8 (per-row) for the Sprapp in-browser WASM engine. int8 is near-lossless for this fp model (ternary post-training-quantization destroyed it — int8 preserves it).
Sample (greedy, in-engine)
Once upon a time, there was a little → girl named Lily. She loved to play outside in the sun and
watch the birds fly high up in the sky. One day, she saw an old man sitting ...
The cat sat on the → tree and said, "I'm sorry I was so sad. Can you forgive me?"
The cat smiled and gave the cat a big hug. From that day on, the cat always shared ...
Files
| file | what |
|---|---|
eeny_int8.knm |
int8 weights, KNM1 v3 (1.76 MB) — for the Sprapp WASM engine |
eeny_final.pt |
full-precision PyTorch checkpoint ({cfg, model}) |
tokenizer.json |
4096-vocab BPE tokenizer |
Limitations
- Domain: TinyStories only (simple children's stories). Not a general LM.
- Tiny: it confabulates and has no world knowledge beyond the TinyStories distribution.
Part of the Sprapp project — offline on-device tiny LMs in the browser (eeny / meeny / miny family). Trained on TinyStories (Eldan & Li, 2023).