Nib 4B (preview), EXL3
EXL3 quants of Nib 4B for exllamav3 and TabbyAPI, made with the exllamav3 1.6 converter on in-domain agent episodes. Each bitrate is its own branch; the main branch holds only this card.
| Branch | Size | Mean KL vs BF16 | Top-1 agreement |
|---|---|---|---|
3.05bpw |
3.6 GB | 0.0257 | 96.3% |
3.55bpw |
3.8 GB | 0.0124 | 97.2% |
4.05bpw |
4.0 GB | 0.0067 | 98.0% |
4.55bpw |
4.2 GB | 0.0046 | 98.5% |
5.05bpw |
4.5 GB | 0.0021 | 99.0% |
6.05bpw |
4.9 GB | 0.0007 | 99.3% |
Bitrates are for the quantized decoder layers. Only the main linear projections are quantized. The vision encoder, embeddings and tied output head, all norms, the Gated DeltaNet gates, convolutions and decay parameters, and the MTP weights are kept byte-for-byte from the BF16 model (in nib-protected.safetensors). Fidelity is measured against the BF16 model on 5,070 held-out assistant tokens from 48 complete agent episodes (up to 32k tokens, ten harnesses), none used for calibration.
4.05bpw is a good default; 5.05bpw and up are nearly indistinguishable from BF16 by these measures. These quants were not run on the browser-task benchmark (the FP8 and NVFP4 builds were, and matched BF16 within noise).
Download
hf download ProCreations/nib-4b-preview-EXL3 --revision 4.05bpw --local-dir nib-4b-preview-EXL3-4.05bpw
Serving (tested: exllamav3 1.6 + TabbyAPI)
model:
model_dir: /path/to/models
model_name: nib-4b-preview-EXL3-4.05bpw
backend: exllamav3
max_seq_len: 32768
cache_size: 32768
reasoning: true
vision: false
Each bitrate passed chat, both thinking modes, reasoning parsing and a parsed tool call through TabbyAPI's OpenAI-compatible API with the bundled chat template. The vision weights are included, but image input through exllamav3 was not tested.
Reasoning effort
Same as Nib 4B: pass chat_template_kwargs
{"enable_thinking": false} for thinking off, or {"enable_thinking": true, "reasoning_effort": "low" | "medium" | "xhigh"}
(xhigh is the default). All our evaluations used temperature 1.0 and top_p 0.95.
About Nib
Nib is Qwen3.5-4B post-trained as a browser agent across common agent harnesses. Benchmarks, usage, training details and limitations (including that it often reports success on tasks it did not finish) are on the main model card. This is a preview release.
License: Apache 2.0, the same as the base model Qwen/Qwen3.5-4B.