ThinkingCap-Qwen3.8-27B — NInfer Artifact Card
ThinkingCap-Qwen3.8-27B is BottleCap AI's token-efficient, thinking-oriented,
multimodal (text + image/video → text) 27B derivative of
Qwen/Qwen3.8-27B. Per BottleCap AI, it
cuts reasoning tokens by 37% on average (11%–66% depending on the benchmark)
while holding an average accuracy of 85.8% against the base model's 86.6%;
on long-context retrieval it cuts reasoning by 39% with accuracy intact
(+2.3pp). It is the second installment of the ThinkingCap series, the successor
to bottlecapai/ThinkingCap-Qwen3.6-27B.
This artifact is a single NVFP4 (W4A4, W8G32-generation) v3 NInfer bundle
of the ThinkingCap checkpoint, intended for serving through the NInfer engine.
It is the ThinkingCap successor to
knoopx/ThinkingCap-Qwen3.6-27B-NInfer,
the Qwen3.6 bundle from the same artifact family.
Scope note. This is a derived inference artifact. The upstream
bottlecapai/ThinkingCap-Qwen3.8-27Brepository (gated) remains the source of truth for the model weights and the canonical distribution. What is described here is a derived, quantized, engine-specific container.
Model Architecture
Architecture properties of Qwen3.8-27B, shared by its finetunes:
| Property | Value |
|---|---|
| Architecture | qwen3_5 (Qwen3_5ForCausalLM) |
| Internal model id | thinkingcap-qwen3.8-27b-nvfp4-bf16-9layers |
| Hidden layers | 64 |
| Attention | hybrid — 16 full-attention (interval 4: layers 3, 7, 11, …, 63) + 48 GDN (recurrent, short convolution width 4) |
| Hidden size | 5120 |
| Intermediate | 17408 |
| Attention heads | 24 query / 4 KV (head dim 256) |
| Vocabulary size | 248,320 |
| Max position | 262,144 |
| Modality | Multimodal (text + image/video input); 27-layer vision encoder (hidden 1152, intermediate 4304, 16 heads, patch 16, out_hidden 5120) |
| MTP / draft | 1 MTP head (mtp_num_hidden_layers = 1) → MTP speculative decoding |
| Base | finetune of Qwen/Qwen3.8-27B |
| Upstream | bottlecapai/ThinkingCap-Qwen3.8-27B (gated) |
| License | Apache-2.0 — see License |
The NInfer Artifact
The single self-contained inference bundle in this repository:
| Property | Value |
|---|---|
| Filename | ThinkingCap-Qwen3.8-27B-nvfp4-w8g32-q4g64-q5g64-q6g64-q8g32-bf16.v3.ninfer |
| Size | 18,324,334,848 bytes (17.06 GiB) |
| SHA-256 | cab598b7f71715dbc1a46dd8832db0c3c30ff457667c940465dd14b78af95ade |
model_id |
thinkingcap-qwen3.8-27b-nvfp4-bf16-9layers |
weights_id |
nvfp4 |
| Bundle generation | v3 |
| Stored objects | 1,307 (1,301 tensors + 6 resources) |
Quantization scheme
NVFP4 W4A4: packed 4-bit FP4 (E2M1) weights with per-tensor group scales
and input_activations: tensor_group, so activations are also 4-bit at
inference. This is a mixed-precision layout: the bulk of the MLP /
full-attention / GDN weight matrices are NVFP4, while a set of control tensors
stays full BF16/FP32, nine text layers are stored BF16, and the vocabulary
endpoints (token embedding, output head) are row-scaled Q8G32.
Weight types present (per the filename format tokens):
| Weight type | Description | Notes |
|---|---|---|
nvfp4 |
4-bit FP4 (E2M1) weights, 4-bit activations (W4A4) | MLP, full-attention, GDN |
q8g32 |
8-bit weights, 32-element group scale | vocabulary endpoints (embedding, head) |
q4g64 / q5g64 / q6g64 |
4/5/6-bit weights, 64-element group | vision towers |
bf16 / fp32 |
Full precision | control weights, 9 text layers, MTP head, scale divisors |
Per-tensor format histogram (verified for this artifact with
tools/artifact/inspect.py): NVFP4 247, Q8G32_F16S 9, Q4G64_F16S 55,
Q5G64_F16S 54, Q6G64_F16S 1, BF16 591, FP32 343, I32 1 — sums to the
1,301 stored tensors. NVFP4 projections run the A4 activation path with stored
scale divisors; all other projections run A16.
Provenance / Conversion
This artifact derives from the gated NVFP4 source
bottlecapai/ThinkingCap-Qwen3.8-27B-NVFP4,
which carries all NVFP4 text projections plus the BF16 vision/MTP/embedding
components and the 9 BF16 text layers.
The bundle was produced as a multi-source graft (byte-repack, zero
conversion / requantization) by the NInfer source converter with a custom
ThinkingCap recipe (graft module tools/convert/sources/graft.py):
- NVFP4 text projections (247 parent objects) — imported encoded from the ThinkingCap NVFP4 checkpoint;
- Q8G32 token embedding / output head, Q4G64 proposal head, I32 token ids,
and the FP32 scale divisors — grafted byte-exact from
kaushikvira/Qwen3.8-27B-thinkingcap-nvfp4full-dflash2-NInfer-v3; - 9 BF16 text layers (attn q/k/v at 3, 7, 11, 15, 19, 23; attn output at 3, 7; GDN output at 4 — 21 tensors) — extracted from the ThinkingCap NVFP4 checkpoint, where these layers are stored BF16; the extraction was validated at 0.995 correlation against the NVFP4-decoded values;
- MTP Q8G32 projections and the vision tower — grafted byte-exact from the
local Swift 1.5 v3 bundle
(
ukisai/Swift-1.5-Qwen3.8-27b-nvfp4-w8g32-q4g64-q5g64-q6g64-bf16.v3.ninfer).
The MTP and Vision components carry base-family values: ThinkingCap's own values for those parameters exist only in NVFP4 form in the checkpoint, and storing them in the profile's target types would have been re-quantization, which was ruled out. The 9 BF16 text layers carry ThinkingCap's own values (stored BF16 in the checkpoint). The MTP, Vision, q8, proposal, and divisor grafts are byte-identical to their sources; the 9 BF16 layers are an extraction validated at 0.995 correlation, not a byte-identity check. The final bundle was re-packed into the family's named-object layout (byte-repack, no weight change).
The embedded frontend uses the Qwen3.8 base native resources (tokenizer,
tokenizer config, generation config, and the two vision preprocessor configs)
together with the maintained NInfer qwen3_8.jinja chat template.
Benchmark reference (upstream checkpoint)
Scores from the upstream release card for the ThinkingCap checkpoint (not this NVFP4 artifact):
| Benchmark | Qwen3.8 base | ThinkingCap | Thinking tokens (base → ours) |
|---|---|---|---|
| GPQA-Diamond | 89.93% | 88.04% | 12,772 → 7,267 (↓ 43.1%) |
| MMLU-Pro | 85.54% | 84.67% | 3,725 → 1,591 (↓ 57.3%) |
| MMMLU | 85.38% | 84.09% | 1,656 → 571 (↓ 65.5%) |
| AIME 2026 | 98.13% | 94.27% | 15,663 → 10,934 (↓ 30.2%) |
| HMMT Feb 2026 | 95.83% | 94.70% | 23,211 → 18,099 (↓ 22.0%) |
Known Issues & Limitations
- No quality benchmark run. No KL-divergence or perplexity comparison has been run on this artifact. Quantization fidelity is not validated by a benchmark. Validation is structural (object inventory, byte-identity of every grafted object) plus a coherent-generation smoke run.
- Grafted base-family values. The MTP projections and the vision tower carry base-family weights, not ThinkingCap values — an inherent consequence of the no-re-quantization constraint. The 9 BF16 text layers carry ThinkingCap's own values (stored BF16 in the checkpoint); they are an extraction (0.995 correlation-validated), not a byte-verified graft.
- Upstream is gated. The ThinkingCap source repositories require granted access; this artifact redistributes the derived bundle only.
License
This NInfer artifact is a derivative of
bottlecapai/ThinkingCap-Qwen3.8-27B
(gated), on top of the Apache-2.0-licensed Qwen/Qwen3.8-27B base. The
artifact is distributed under the Apache License 2.0.
Users remain responsible for complying with the license and applicable laws.
Citation / Provenance
Upstream repo:
bottlecapai/ThinkingCap-Qwen3.8-27B(gated)NVFP4 source:
bottlecapai/ThinkingCap-Qwen3.8-27B-NVFP4(gated)Graft source (q8 endpoints, proposal head, divisors):
kaushikvira/Qwen3.8-27B-thinkingcap-nvfp4full-dflash2-NInfer-v3Graft source (MTP, Vision) and layout reference:
ukisai/Swift-1.5-Qwen3.8-27bv3 bundleBase:
Qwen/Qwen3.8-27BFamily predecessor:
knoopx/ThinkingCap-Qwen3.6-27B-NInferInference engine: NInfer
- Downloads last month
- 91
Model tree for knoopx/ThinkingCap-Qwen3.8-27B-NInfer
Base model
Qwen/Qwen3.8-27B