darrellbest/Hemmingway-1-GGUF

Hemmingway-1 GGUF

GGUF builds of Altworld/Hemmingway-1 (Qwen3.8-27B fine-tune for everyday writing), made from darrellbest/Hemmingway-1-VL, the same weights in the stock Qwen3.8 layout with the vision tower restored. All credit for the model goes to Altworld.

File	Precision	Size
Hemmingway-1-BF16.gguf	BF16, lossless, incl. the MTP head (nextn_predict_layers = 1)	54.66 GB
Hemmingway-1-Q8_0.gguf	Q8_0	29.05 GB
Hemmingway-1-Q6_K.gguf	Q6_K, imatrix	22.43 GB
Hemmingway-1-Q4_K_M.gguf	Q4_K_M, imatrix	16.81 GB
Hemmingway-1-mmproj-BF16.gguf	vision projector, BF16, lossless	0.93 GB

All files keep the MTP head. The imatrix came from ~100K tokens of UltraChat training conversations rendered with the model's own chat template.

Quant quality

llama-perplexity --kl-divergence against the BF16 logits, on held-out chat (UltraChat test conversations, ~24K tokens). "Same top token" is how often the quant's most likely next token equals BF16's.

Build	Same top token	Mean KLD	99th pct KLD	PPL (BF16: 3.227)
Q8_0	98.9%	0.0076	0.037	3.246
Q6_K	98.3%	0.0116	0.085	3.218
Q4_K_M	95.9%	0.0479	0.715	3.221

Every quant was also run in llama-server (CUDA). Each answered correctly and described a held-out photo through the mmproj. All three produced the same get_weather tool call as the transformers bf16 reference.

Verification of the BF16 files

Every one of the 866 GGUF tensors was mapped back to its safetensors source and checked, independently of the converter:

	tensors	result
copied	266	byte-identical
rearranged by the converter (V-head reorder, conv squeeze)	384	exactly the same values, only reordered
RMSNorm weights, stored as 1 + w (llama.cpp convention)	168	exact
A = -exp(A_log) (llama.cpp convention)	48	bit-identical to what transformers computes at runtime

In the mmproj, 332 vision tensors hold the identical values. The 3D patch-embedding convolution is split by llama.cpp into its two temporal halves, and both are exact.

Engines don't produce bit-identical logits (different kernels). As a control, transformers diverges from itself (sdpa vs eager attention) on the same near-tied tokens. llama.cpp BF16 matched transformers' greedy output for 241/543 tokens before divergence, vs 214/543 for transformers-vs-transformers.

Runtime check (llama-server, CUDA): correct text answers, and an accurate image description through the mmproj.

Use

llama-server -m Hemmingway-1-BF16.gguf --mmproj Hemmingway-1-mmproj-BF16.gguf --jinja -ngl 99

It's a thinking model. Pass "chat_template_kwargs": {"enable_thinking": false} to answer directly.

Converted with llama.cpp bd4f514 (+ a one-line fix for mtp_num_hidden_layers: null, not needed for this model).

Licence

CC BY-NC 4.0, as Hemmingway-1: non-commercial use, with credit to Hemmingway-1 / Altworld. Commercial use: luka@hemmingway.io.