kybrcore/Hemmingway-1-NInfer

Hemmingway-1 — NInfer v3 artifact

Exploration release. Published to demonstrate that a third-party finetune can be quantized into the NInfer v3 format with the upstream groupwise-int recipe. It is not part of this publisher's benchmarked model line: quality suites and independent kit-verification are deliberately not planned (see Quality). Use it freely; do not read it as a vetted release.

Single-file NInfer v3 artifact of Altworld/Hemmingway-1 (an everyday-writing finetune of Qwen3.8-27B), re-containerized for the NInfer v3 runtime with the MTP head, DFlash2 drafter and the indexed proposal head assembled in.

Property	Value
File	hemmingway_1.ninfer
Size	20,141,703,168 bytes (~18.76 GiB)
SHA256	5b1eefbff237c30bfac59444c83a1fb70b4f0a8b79346e931eb077d570cd8e33
Format	NInfer artifact schema v3 (single file, embedded chat template)
Weights	≈16.7 GiB materialized on device (MTP lane); ≈18.3 GiB with the DFlash2 lane
Context	262,144 tokens (native)
Modalities	text only — this finetune has no vision tower

This is a quantization/re-containerization, not a new training run. All stored codes are computed at conversion time from the pinned BF16 finetune by the upstream qwen3_8_27b groupwise-int recipe. See Composition.

Composition

Conversion used the upstream NInfer v3 converter (tools/convert) with the built-in qwen3_8_27b recipe — the same recipe that produces the official neroued/Qwen3.8-27B-NInfer artifact (revision 1cbd84e7221e51186bd7f093a149912d2489625b). Projections are quantized with per-group max-abs groupwise quantization; activation policy is A16Only throughout (weight-only quantization).

Component	Encoding	Source
Text projections, Q4 roles (attention q/k, gdn q/k, mlp gate/up)	q4_g64_fp16	Hemmingway-1 BF16
Text projections, Q5 roles (attention o, gdn z/value/output, mlp down)	q5_g64_fp16	Hemmingway-1 BF16
GDN a/b control gates (96 tensors)	BF16 (unquantized)	Hemmingway-1 BF16
Convolutions, all norms, A_log / dt_bias	BF16 / FP32	Hemmingway-1 BF16
token_embedding / output_head	q8_g32 groupwise, computed locally	Hemmingway-1 BF16
MTP head (input projection + 1 block)	q8_g32 groupwise	Hemmingway-1 BF16
DFlash2 companion (5 blocks + feature projection, W8G32)	q8_g32 groupwise	z-lab BF16 drafter
Proposal head	Q4 groupwise, frequency-ranked	derived from output_head
Chat template	upstream maintained qwen3_8.jinja (embedded; thinking on, effort xhigh)	Neroued/ninfer

Stored object counts: 851 tensors in 855 objects — q4_g64_fp16 129, q5_g64_fp16 192, q8_g32_fp16 28, bf16 405, fp32 96, int32 1.

Tokenizer note

Hemmingway-1's published tokenizer.json declares the older Qwen2 pre-tokenizer ([\p{L}]+) and omits added_tokens_decoder; the NInfer Qwen frontend validates one exact pipeline and refuses to load it. The artifact therefore embeds the upstream Qwen3.8 tokenizer resource declaration. This is a declaration fix, not a vocabulary swap: the two tokenizers have byte-identical vocab (248,044 entries), merges (247,587) and added_tokens (33), and derive the same token domain (248,077 tokens, same 21 special ids). The finetune's own generation_config.json is preserved, so stop-token semantics are unchanged.

Serving

Requires an NInfer v3 runtime — the upstream Neroued/ninfer engine serves this artifact as-is. 64-bit Linux, CUDA toolkit with sm_120a, one RTX 5090 (32 GB).

Tested engine revision: Neroued/ninfer @ 5b4303c0ea0e8ab2be3efa54a677829f3edab6e5. This artifact was built and validated against that commit; use it, or a revision whose conversion and kernel paths are known-compatible with it. The artifact format is not versioned against later engine releases, so a newer upstream revision is not guaranteed to load or execute this file — pin the engine, not just the artifact.

git clone https://github.com/Neroued/ninfer
cd ninfer && git checkout 5b4303c0ea0e8ab2be3efa54a677829f3edab6e5

# native MTP=3 speculation, native 262K context
ninfer-serve hemmingway_1.ninfer \
  --model-id hemmingway-1 \
  --spec mtp --draft-tokens 3 --lm-head-draft \
  --max-context 262144 --kv-dtype nvfp4 --max-concurrency 3 \
  --host 0.0.0.0 --port 30000

# alternatively the DFlash2 companion embedded in the same file
ninfer-serve hemmingway_1.ninfer \
  --spec dflash2 --draft-tokens 7 --lm-head-draft \
  --max-context 262144 --kv-dtype nvfp4

Do not pass --vision: this artifact contains no vision component, and the loading phase fails if one is requested.

Memory, measured on a single RTX 5090 32 GB with NVFP4 KV and native 262K context (auto-sized by the engine):

Speculation	Weights	KV pool (auto)	Runtime	GPU used
MTP=3	16.7 GiB	652,288 tokens	13.2 GiB	30,910 MiB
DFlash2=7	18.3 GiB	510,912 tokens	11.5 GiB	29,760 MiB

Because there is no vision tower, the whole fixed allocation goes to KV — the pool is larger than the vision-enabled Qwen3.8-27B artifacts on the same card.

Quality

Not benchmarked by design. This artifact is an exploration release: the goal was to test whether a third-party finetune can be quantized into the NInfer format with the upstream groupwise-int recipe. That question is answered by the artifact loading and serving correctly, which it does. The publisher's standard evaluation suites (AIME / GPQA-Diamond / IFBench / NIAH, and the upstream serving-corpus draft-acceptance method) are deliberately not planned for this release, and no independent kit-verification run is claimed.

Treat every number below as single-request serving smoke results, not benchmark scores.

Probe	MTP=3	DFlash2=7
Decode throughput	~170 tok/s (81-token generation)	119–137 tok/s
Draft acceptance (smoke)	52.7% (93 drafted / 49 accepted)	23–29%
Instruction following	correct (84*3/2 → 126)	same weights

A quality section will be added once the benchmark suites have been run.

Lineage and attribution

Qwen/Qwen3.8-27B                        (BF16 base, Apache-2.0)
  └─ Altworld/Hemmingway-1              (finetune, Apache-2.0)
       ├─ z-lab/Qwen3.8-27B-DFlash2     (DFlash2 drafter, Apache-2.0)
       └─ Neroued/ninfer @ 5b4303c0     (v3 engine, converter, artifact format)

• Hemmingway-1 is by Altworld; its card describes it as built for everyday writing (messages, emails) and reports its own CommunicationBench / Human-Likeness results. Those numbers belong to the finetune and are unaffected by this re-containerization.
• DFlash2 is by Inco AI (block-diffusion drafter, ICML 2026); this artifact embeds the drafter weights quantized to W8G32 as the speculative companion.
• The v3 artifact format, converter and all kernel paths are from Neroued/ninfer (upstream).
• This repository is an independent packaging of these weights for the upstream runtime; it is not an official release by the upstream engine maintainers, by Altworld, or by Inco AI.

Provenance notes

Every external input is pinned to an immutable revision. Source weights were downloaded and verified file-by-file against their published SHA-256 digests.

Role	Repository	Revision
Finetune weights (base of this artifact)	Altworld/Hemmingway-1	4d711aac0f0043075ae334d2a3de3db3e10135c9
DFlash2 drafter companion	z-lab/Qwen3.8-27B-DFlash2	50307d4c4cde6860d4eee73e2547cd786fe8e8a4
Tokenizer declaration (embedded resource)	Qwen/Qwen3.8-27B	1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
Converter, recipe, artifact format	Neroued/ninfer (GitHub)	5b4303c0ea0e8ab2be3efa54a677829f3edab6e5
Reference artifact for the recipe comparison	neroued/Qwen3.8-27B-NInfer	1cbd84e7221e51186bd7f093a149912d2489625b

Embedded resource digests: tokenizer.json = 0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3, tokenizer_config.json = b11349aafa7cdc6a320767cf7ceb29ed82f7eda5d65e8e0819e76f0ce947bf27.

Conversion: recipe qwen3_8_27b, --device cpu, components text,mtp,dflash2, --proposal (131,072 rows), chat template tools/chat_templates/qwen3_8.jinja. Wall time 1,131 s. hemmingway_1.ninfer.conversion.json (shipped alongside) records every per-tensor method, format, source and layout chosen by the run.

Reproduction kit: conversion/ in this repository.

Caveats

• The MTP head is the finetune's own (trained by Altworld); the DFlash2 companion is a separate drafter that was trained against the Qwen3.8-27B base, not against this finetune. Expect its acceptance rate to be lower than on the base model — on this artifact it measured ~23–29% against ~53% for the artifact's own MTP head. Prefer MTP unless you measure otherwise.
• Serving numbers above come from one RTX 5090 and the upstream engine revision 5b4303c0; they are reproducibility anchors, not vendor-published figures.
• Text only: no image or video input.