mys/kev-0.8b-GGUF

Kev 0.8B GGUF (ggmlc)

English System 1 decision model compiled from jaredpalmer/kev-0.8b (Qwen3.5-0.8B hybrid Gated DeltaNet, LoRA merged in fp32 before export, context 2048).

These files are not llama.cpp / llama-cli GGUFs. They are produced by ggmlc. Loading them in llama.cpp will fail.

Typed questions (choice / score / noul) are scored in one forward. There is no autoregressive token generation. Tokenizer, sequence program, and postprocess live in the GGUF (ggmlc.decision).

Source and CLI: examples/laya

Smaller attention-only sibling: kev-0.5b-GGUF

Files

File	Quant	Size	Notes
kev_0.8b_f16.gguf	F16	~1.46 GB	Best numerical match. Qwen3.5 zero-centered RMS is fused.
kev_0.8b_q8_0.gguf	Q8_0	~790 MB	Uniform Q8_0. 1D norms stay F32.
kev_0.8b_ud_q4_k_m.gguf	UD_Q4_K_M	~914 MB	Embedding stays F16 (248k vocab), sensitive projections Q8_0, other weights Q4_0. Larger than uniform Q8_0 because the embedding is not quantized.

huggingface-cli download mys/kev-0.8b-GGUF kev_0.8b_f16.gguf --local-dir .

Run with laya

laya info kev_0.8b_f16.gguf
laya decide kev_0.8b_f16.gguf --preset email --device auto --cuda-graph
laya serve kev_0.8b_f16.gguf --port 8080 --device auto --cuda-graph
laya bench kev_0.8b_f16.gguf --preset email --device auto --cuda-graph

serve is Decision Studio plus POST /v1/systemone (TypeSafe). The advertised model id is kev-0.8b.

License

Apache 2.0, same as jaredpalmer/kev-0.8b. Compiler: ggmlc (MIT).