anonymus-1bit-gpt — the first 1-bit "Hungarian" LLM

A byte-level ternary GPT trained on the Gesta Hungarorum of Anonymus (P. dictus magister, notary of King Béla, c. 1200) — the founding chronicle of the Hungarian nation. Every weight is a trit {−1, 0, +1}; the deployed model is bit-packed at ~1.6 bits/weight and ships as a 196 KB checkpoint.

Built entirely on ultragraph, a pure-Python (+numpy) library where the byte-graph is the 1-bit LLM.

The honest part

Anonymus wrote in medieval Latin, not Hungarian — so this is a model of the language the Hungarian origin story was written in, studded with the Hungarian names that survive untranslated (hetumoger, almus, arpad, scithia, hungarij). The corpus is the unambiguously public-domain Latin original, 94 KB, the entire Gesta and nothing else. Data is truth.

Specs

params	384,448 (ternary weights + fp32 embedding/norms)
weights	ternary {−1, 0, +1}, ~1.58 bits each
tokenizer	byte-level, vocab 256 (lossless UTF-8)
arch	GPT(d_model=96, n_layers=3, n_heads=4) — RoPE + KV-cache
training	Adam + cosine schedule, ~1500 steps, single CPU; loss 5.84 → 1.78
on disk	196 KB (bit-packed deployed checkpoint)

Usage

pip install ultragraph-1bit

from huggingface_hub import hf_hub_download
from ultragraph import GPT, ByteTokenizer

path = hf_hub_download("PeetPedro/anonymus-1bit-gpt", "anonymus.gpt.npz")
tok = ByteTokenizer()
m = GPT.load_deployed(path)   # runs straight from the ternary bytes, no fp32 master
print(tok.decode(m.generate(tok.encode("Almus dux "), n_new=120, temperature=0.8, top_p=0.9)))

Samples

Prompted, temperature 0.8 — it has never seen a word outside the Gesta:

Almus dux fuersis marpalere dilitutima uenerunt, dux cum patis se suom an secitum terras suis sanon …

In terra scithica megaturimos sames et controum perse hundiam fodias, sed fluuium et terre uidiutater … pro temor hungat …

Not coherent — 384k params from 94 KB — but it caught the notary's register: dux (leader) where Anonymus puts it, fluuium et terre ("river and land," the substance of a conquest chronicle), the reach toward hungarie.

Reproduce

uv run python examples/fetch_gesta.py     # pull + clean the corpus
uv run python examples/anonymus_lm.py     # train -> deployed 196 KB checkpoint

Links

• Library: https://github.com/peterlodri-sec/ultra-graph
• PyPI: https://pypi.org/project/ultragraph-1bit/
• Write-up: https://pocoo.vaked.dev/posts/2026-07-11-first-1bit-hungarian-llm-anonymus
• Source text (public domain): https://la.wikisource.org/wiki/Gesta_Hungarorum

License

MIT. Trained on the public-domain Latin Gesta Hungarorum.