spinozans/emender-e97-4b-gguf

EMENDER E97 4B — GGUF (llama.cpp)

GGUF conversions of the EMENDER E97 4B model family for llama.cpp. E97 is a 4,045,972,080-parameter hybrid recurrent architecture (bounded tanh-state sequential recurrence, 18 layers, d_model 3840, 60 heads x 64, SwiGLU MLP hidden 9600, tied embedding/lm_head, vocab 50,281 p50k_base + 24 codex space-run extras). It has NO attention and NO KV cache — state is bounded per layer, which is why it runs efficiently on CPU.

Two checkpoints are provided:

1. e97-4b-base-99B-f32 / e97-4b-base-99B-q4_0 — the raw from-scratch pretraining checkpoint (~99.7B tokens). Pure LM continuation model; no post-training, no chat training.
2. e97-4b-pi-f32-aligned / e97-4b-pi-q8_0-v2 / e97-4b-pi-q4_0 — the verified Pi-native tool-agent checkpoint (the dual-gate-passed, promoted v6-u96), post-trained for a coding-agent tool surface.

Quantification — read before choosing

Every quant was qualified on a frozen 32-prompt panel against its own f32 reference (top-1 next-token agreement, KL over top-64 logits, greedy-exact):

Model	Quant	top-1 vs f32	mean KL	size	decode tok/s*	Verdict
agent	q8_0-v2	32/32 (100%)	0.0020	4.87 GB	19.4	qualified — recommended
agent	q4_0	31/32 (96.9%)	0.0371	2.94 GB	25.4	footprint option — one prompt off
base-99B	q4_0	27/32 (84.4%)	0.0186	2.94 GB	25.0	footprint option

* 32 CPU threads, shared host. f32 runs at ~7-8 tok/s.

Honest guidance: q8_0-v2 is the smallest quant that passes the port's qualification precedent (top-1 >= 0.97, KL <= 0.05). The q4_0 files are provided as small-footprint options with their measured degradation stated — the agent q4_0 misses by exactly one prompt; the base q4_0 degrades more.

Running (requires the emender_e97 llama.cpp arch)

These GGUFs use a registered custom architecture (emender_e97); use a llama.cpp build with that arch registered (the Emender repo carries the registration patch series). Standard invocation:

llama-cli -m e97-4b-base-99B-q4_0.gguf -p "Once upon a time" -n 64 -t 32 --temp 0.7 -st
llama-server -m e97-4b-pi-q8_0-v2.gguf --host 127.0.0.1 --port 8151 -t 32

Caveats, stated plainly

• No chat template is embedded. llama.cpp falls back to a ChatML-style wrapper; that is NOT the trained format. The agent checkpoint was trained on a strict five-line Analysis/Commentary/Action/Arguments frame protocol under a canonical system prompt — for on-distribution behavior, use the Emender repo's serving path (OpenAI-compatible server) or the CPU REPL, which implement the trained codec.
• Greedy decoding repetition-collapses this model (both checkpoints; a model property, reproduced identically across f32 and q4). Sample with temperature ~0.7 / min-p, or use DRY/repetition penalties.
• The base model is a raw LM: it continues text, it does not chat.
• The agent checkpoint is a narrow coding/tool agent, not a general assistant.

Provenance

• Base: from-scratch frontier pretraining run, step 24448 (~99.72B tokens), checkpoint SHA-256 3ace004251643acf2e7c7f720e8f29968ad0a483441553c0c885b87b3df84568.
• Agent: promoted v6-u96 dual-gate-passed checkpoint, SHA-256 d81464982c3ebc0d72769a87e079068bf535d6ca2010185dc1bb03ce264b8f5b.
• Conversion: bitwise-verified tensor-by-tensor against the source state dicts.
• Port qualification: GGML runner + registered llama.cpp arch vs GPU reference: 32/32 top-1, KL 0.0007 (f32).