Tinfield 1 Compact
Tinfield 1 Compact is Tinfield 1 compressed to run on hardware you can actually buy. 330 GB of BF16 weights down to 72 GB, with 45.2 GiB resident on the GPU, so a 177B agentic model fits in the space a 70B usually takes.
Mean KL against the Q8 reference is 0.287. The recipe puts bits where they change the output: expert gate and up projections at IQ2_XXS, expert down projections and the engram table at IQ4_NL, token embedding and output head at Q6_K, the rest of the spine at Q8_0, routers untouched at F32.
| File size | 72.0 GiB |
| Weights resident on GPU | 45.2 GiB |
| Smaller than BF16 | 4.6x by file, 5.2x on the GPU side |
| Mean KL against Q8 | 0.287 |
Running it
llama.cpp must be at commit 972d2313 or later. This architecture is not in older builds, and they
fail to load the file rather than warning about it. If the model will not open, check this first.
The six parts are a split GGUF: point llama.cpp at the first part and it finds the rest.
llama-server -m tinfield-1-compact-IQ2XXS-00001-of-00006.gguf --jinja \
--temp 1.0 --top-p 0.95 --top-k 20 \
-fa on -ctk q8_0 -ctv q4_0 \
-ngl 999 -c 131072
--jinja is required or the chat template is ignored and tool calling breaks. The sampler values are
Qwen's own, from generation_config.json. The KV flags cost 9,984 bytes per token of context instead of
24,576, and that cost is what the KL number above accounts for.
Skip n-gram self-speculation. On agent traffic it accepted only 16 to 25% of drafted tokens, which costs more verification work than it saves. Output is identical either way.
Fitting it on your hardware
| Setup | Add |
|---|---|
| 64 GB Mac, unified memory | nothing, the default above |
| 32 GB GPU plus system RAM | --n-cpu-moe 24 -c 65536 |
| 16 GB GPU plus 64 GB RAM | --cpu-moe -c 65536 |
| 32 GB RAM, experts from NVMe | --cpu-moe -c 32768 |
Experts are 40.4 GiB (0.84 GiB per layer), the spine is 4.8 GiB, and the engram table is 26.8 GiB and always memory-mapped on the CPU side, since a token reads only a handful of its rows.
On the NVMe tier, do not pass --mlock or --no-mmap. The OS cache is doing the work: about 16 GB of
expert cache catches roughly 80% of routings.
On macOS, 45.2 GiB on the GPU side exceeds the default wired limit once context is added. Raise it before launching:
sudo sysctl iogpu.wired_limit_mb=56000
launch_presets.sh in this repo wraps all four tiers.
Worth knowing
Reasoning is preserved by default. The chat template keeps prior turns' thinking in context and
llama.cpp enables that automatically, so on long agent runs context fills faster than you expect. Pass
--no-reasoning-preserve to turn it off.
-c is total context, not per slot. With -np 8 -c 16384 each slot gets 2048 tokens and requests
truncate without saying so. Multiply: -np 5 -c 655360 gives five slots of 131072.
First load takes about 35 seconds from fast local storage, longer over a network mount. It is not hung.
Quality
Mean KL divergence against the Q8 build of the same weights is 0.287, measured on 22 held-out session slices at 4,096 context with the deployment KV setting.
The benchmark scores on the parent model are for the BF16 weights. This build has not been evaluated on Terminal-Bench 4.0 or DeepSWE, so treat those numbers as the ceiling rather than a measurement of this file.
A smaller build is at Tinfield 1 Mini.
License
Qwen Community License 1.0, following the base model.
- Downloads last month
- 228
We're not able to determine the quantization variants.