Instructions to use bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="bottlecapai/ThinkingCap-Qwen3.6-27B-FP8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("bottlecapai/ThinkingCap-Qwen3.6-27B-FP8")
model = AutoModelForMultimodalLM.from_pretrained("bottlecapai/ThinkingCap-Qwen3.6-27B-FP8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "bottlecapai/ThinkingCap-Qwen3.6-27B-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.6-27B-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/bottlecapai/ThinkingCap-Qwen3.6-27B-FP8

SGLang

How to use bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "bottlecapai/ThinkingCap-Qwen3.6-27B-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.6-27B-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "bottlecapai/ThinkingCap-Qwen3.6-27B-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.6-27B-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use bottlecapai/ThinkingCap-Qwen3.6-27B-FP8 with Docker Model Runner:
```
docker model run hf.co/bottlecapai/ThinkingCap-Qwen3.6-27B-FP8
```

bottlecapai/ThinkingCap-Qwen3.6-27B-FP8

FP8 quantization of bottlecapai/ThinkingCap-Qwen3.6-27B — capability of Qwen3.6-27B with 50% less thinking tokens on average, achieved by finetuning Qwen3.6-27B (Qwen Team, 2026) with online reinforcement learning while preserving the original answer quality and style.

➡️ Full model description, evaluation results (multi-seed, statistically tested), recommended sampling params, and citation: see the main model card at bottlecapai/ThinkingCap-Qwen3.6-27B.

About this quantization

Weights are stored as FP8 (E4M3) with per-channel scales; activations are quantized dynamically per token at inference (FP8_DYNAMIC W8A8, produced data-free with llm-compressor) in the compressed-tensors format that vLLM and SGLang load natively. ≈29 GB instead of ≈55 GB bf16 — near-lossless quality at half the memory, with FP8 tensor-core throughput on Ada / Hopper / Blackwell GPUs (older GPUs serve it via weight-only FP8-Marlin kernels).

Kept in bf16: lm_head, MTP heads, and the vision tower.

Usage (vLLM)

vllm serve bottlecapai/ThinkingCap-Qwen3.6-27B-FP8

For local llama.cpp / Ollama / LM Studio use, see the GGUF quantizations at bottlecapai/ThinkingCap-Qwen3.6-27B-GGUF.

Expected performance

From our internal serving-validation harness (vLLM, single-stream, temperature 0) on a fast N=100/dataset subset of MMLU-Pro (reasoning) and RealWorldQA (vision) — a quick quant-parity + decode-speed check, not the headline accuracy evals (for the multi-seed, statistically-tested results see the main model card).

Two things to read off it: FP8 matches the bf16 finetune's accuracy within subset noise while decoding ≈34% faster (85 vs 64 tok/s), and MTP self-speculative decoding (≈3.35 accepted tokens per verify step) adds a further ≈2.4× — stacking with the finetune's token savings to ≈7× faster per task than the unquantized base.

median tokens = median completion length (the finetune's lever); task s = median tokens ÷ single-stream tok/s (real per-request time); speedup is vs the base model in standard decoding. NVIDIA NVFP4 is NVIDIA's ModelOpt NVFP4 quant of the base model, shown for comparison.

MMLU-Pro (reasoning)

config	acc	median tokens	tok/s	task s	speedup	accept_len
Qwen3.6-27B base · standard	0.86	1936	62.0	31.2	1.00×	—
Qwen3.6-27B base · MTP	0.83	2042	160.6	12.7	2.46×	3.32
ThinkingCap bf16 · standard	0.86	894	63.6	14.1	2.21×	—
ThinkingCap bf16 · MTP	0.87	884	164.2	5.4	5.78×	3.34
ThinkingCap-FP8 · standard	0.91	876	85.4	10.3	3.03×	—
ThinkingCap-FP8 · MTP	0.89	893	208.1	4.3	7.26×	3.35
NVIDIA NVFP4 (base) · standard	0.84	1842	98.4	18.7	1.67×	—
NVIDIA NVFP4 (base) · MTP	0.82	1980	239.9	8.3	3.76×	3.30

RealWorldQA (vision)

config	acc	median tokens	tok/s	task s	speedup	accept_len
Qwen3.6-27B base · standard	0.71	601	62.0	9.7	1.00×	—
Qwen3.6-27B base · MTP	0.69	545	160.6	3.4	2.85×	3.32
ThinkingCap bf16 · standard	0.76	272	63.6	4.3	2.26×	—
ThinkingCap bf16 · MTP	0.80	277	164.2	1.7	5.71×	3.34
ThinkingCap-FP8 · standard	0.80	265	85.4	3.1	3.13×	—
ThinkingCap-FP8 · MTP	0.79	290	208.1	1.4	6.93×	3.35
NVIDIA NVFP4 (base) · standard	0.67	535	98.4	5.4	1.80×	—
NVIDIA NVFP4 (base) · MTP	0.67	537	239.9	2.2	4.41×	3.30

Downloads last month: 3,184

Safetensors

Model size

27B params

Tensor type

BF16

F8_E4M3

Model tree for bottlecapai/ThinkingCap-Qwen3.6-27B-FP8

Base model

Qwen/Qwen3.6-27B

Finetuned

bottlecapai/ThinkingCap-Qwen3.6-27B

Quantized

(15)

this model