Instructions to use bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M

Ollama
How to use bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF with Ollama:
```
ollama run hf.co/bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M
```
Unsloth Desktop

How to use bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF with Docker Model Runner:
```
docker model run hf.co/bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M
```

Lemonade

How to use bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.ThinkingCap-Qwen3.8-27B-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF

GGUF / llama.cpp quantizations of bottlecapai/ThinkingCap-Qwen3.8-27B — the ThinkingCap finetune of Qwen3.8-27B that keeps the base model's answer quality while reasoning in far fewer tokens.

➡️ Full model description, evaluation results (multi-seed, statistically tested), the measured thinking-token reduction, recommended sampling params, and citation: see the main model card at bottlecapai/ThinkingCap-Qwen3.8-27B.

About GGUF and quantization

GGUF is a single-file model format for running LLMs locally with llama.cpp and compatible runtimes (Ollama, LM Studio, …). The quantized variants below store weights at reduced precision — e.g. ≈5.1 bits per weight for Q4_K_M instead of the 16-bit f16 source — cutting download size and memory severalfold at a small, measured quality cost.

The low-bit files are built with an importance matrix (activation statistics from a chat-templated calibration corpus) and a per-tensor precision layout: the attention projections of the full-attention layers and the linear-attention output projections stay at 6–8 bit while the feed-forward weights take the 4-bit hit.

Files

File	Quant	Size
`ThinkingCap-Qwen3.8-27B-IQ4_XS.gguf`	IQ4_XS	15.5 GB
`ThinkingCap-Qwen3.8-27B-Q4_K_M.gguf`	Q4_K_M	17.4 GB
`ThinkingCap-Qwen3.8-27B-Q6_K.gguf`	Q6_K	23.9 GB
`ThinkingCap-Qwen3.8-27B-Q8_0.gguf`	Q8_0	29.0 GB
`ThinkingCap-Qwen3.8-27B-f16.gguf`	f16	54.7 GB
`mmproj-ThinkingCap-Qwen3.8-27B-f16.gguf`	mmproj (vision)	931 MB

f16 is the unquantized source; Q8_0 and the smaller Q6_K showed no visible accuracy loss in our screen; Q4_K_M is the recommended size/quality balance for most local setups; IQ4_XS is the smallest variant that still tracks the f16 model closely.

Usage (llama.cpp)

# pull a specific quant straight from the Hub and chat
llama-cli -hf bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M -p "Hi"

# or download one file and run it
huggingface-cli download bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF ThinkingCap-Qwen3.8-27B-Q4_K_M.gguf --local-dir .
llama-cli -m ThinkingCap-Qwen3.8-27B-Q4_K_M.gguf -p "Hi"

Use the sampling settings from the main model card (the base model's recommended thinking-mode settings). Greedy decoding can loop; keep temperature at the recommended value.

Speculative decoding (MTP)

These GGUFs carry the model's MTP (multi-token-prediction) head, so llama.cpp can run self-speculative decoding for a decode speed-up — no separate draft model needed. Add --spec-type draft-mtp when serving:

llama-server -hf bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF:Q4_K_M --spec-type draft-mtp --spec-draft-n-max 3

Requires a llama.cpp build with MTP support for this architecture (v0.4.1 or newer). It speeds up decoding at 4 parallel slots (see Decode speed and MTP under Expected performance); larger batches are untested. Runtimes that predate MTP support for this architecture may refuse to load the file (missing tensor blk.64…) — update the runtime.

Vision (image input)

ThinkingCap is a vision-language model. Image input needs the multimodal projector mmproj-ThinkingCap-Qwen3.8-27B-f16.gguf (in this repo) loaded alongside a text GGUF — the single f16 mmproj pairs with any of the quants above.

LM Studio / Jan / Ollama, …: download the mmproj-*.gguf from this repo; LM Studio auto-detects it and enables the image (🖼️) button.
llama.cpp CLI:

huggingface-cli download bottlecapai/ThinkingCap-Qwen3.8-27B-GGUF \
  ThinkingCap-Qwen3.8-27B-Q4_K_M.gguf mmproj-ThinkingCap-Qwen3.8-27B-f16.gguf --local-dir .
llama-mtmd-cli -m ThinkingCap-Qwen3.8-27B-Q4_K_M.gguf \
  --mmproj mmproj-ThinkingCap-Qwen3.8-27B-f16.gguf --image photo.jpg -p "Describe this image."

llama-server: add --mmproj mmproj-ThinkingCap-Qwen3.8-27B-f16.gguf to expose an OpenAI-compatible vision endpoint.

Expected performance

Accuracy and completion length against the bf16 weights on our quantization screen. Every row answers the same question subsets of five benchmarks, thinking at the chat template's default reasoning effort (xhigh), with sampled decoding (temperature 1.0, top_p 0.95, top_k 20, min_p 0.0) and a 65,536-token generation cap. The GGUFs are served by llama.cpp with the mmproj loaded for the RealWorldQA images; the bf16 reference is served by vLLM. acc is the mean over seeds, and the ± is the 95% interval over questions (each question averaged over its seeds first), so with 20–100 questions per benchmark it spans several points. AA-LCR answers are graded by Gemma-4-26B-A4B-it. median tokens / mean tokens = completion length (reasoning plus answer) over the questions; for GPQA-Diamond, the median and mean of the two seeds' values.

No quant differs from bf16 by more than that interval on any benchmark. This is a screen — small subsets, one seed on four of the five benchmarks — so read it as "no visible accuracy loss", not as a measured equivalence.

MMLU-Pro (reasoning) — 100 questions × 1 seed

config	acc (mean ± 95% CI)	median tokens	mean tokens
ThinkingCap-Qwen3.8-27B bf16 (vLLM)	0.850 ± 0.070	144	1469
f16	0.870 ± 0.066	135	1508
Q8_0	0.880 ± 0.064	138	1448
Q6_K	0.840 ± 0.072	129	801
Q4_K_M	0.850 ± 0.070	147	1068
IQ4_XS	0.840 ± 0.072	157	1279

RealWorldQA (vision) — 100 questions × 1 seed

config	acc (mean ± 95% CI)	median tokens	mean tokens
ThinkingCap-Qwen3.8-27B bf16 (vLLM)	0.830 ± 0.074	111	537
f16	0.830 ± 0.074	112	589
Q8_0	0.840 ± 0.072	107	666
Q6_K	0.790 ± 0.080	108	350
Q4_K_M	0.810 ± 0.077	102	370
IQ4_XS	0.810 ± 0.077	106	538

GPQA-Diamond (graduate-level science) — 60 questions × 2 seeds

config	acc (mean ± 95% CI)	median tokens	mean tokens
ThinkingCap-Qwen3.8-27B bf16 (vLLM)	0.833 ± 0.088	1191	8378
f16	0.875 ± 0.072	1084	7357
Q8_0	0.858 ± 0.084	970	7773
Q6_K	0.867 ± 0.076	1160	7401
Q4_K_M	0.867 ± 0.076	1352	6327
IQ4_XS	0.900 ± 0.072	883	7025

IFBench (instruction following) — 40 questions × 1 seed

config	acc (mean ± 95% CI)	median tokens	mean tokens
ThinkingCap-Qwen3.8-27B bf16 (vLLM)	0.700 ± 0.142	2039	6013
f16	0.725 ± 0.138	1929	5020
Q8_0	0.650 ± 0.148	2124	5103
Q6_K	0.650 ± 0.148	2129	4657
Q4_K_M	0.725 ± 0.138	2914	4886
IQ4_XS	0.725 ± 0.138	2725	4844

AA-LCR (long-context reasoning) — 20 questions × 1 seed

config	acc (mean ± 95% CI)	median tokens	mean tokens
ThinkingCap-Qwen3.8-27B bf16 (vLLM)	0.650 ± 0.209	745	1587
f16	0.650 ± 0.209	828	1526
Q8_0	0.650 ± 0.209	1359	1358
Q6_K	0.700 ± 0.201	1100	1634
Q4_K_M	0.750 ± 0.190	1098	1358
IQ4_XS	0.700 ± 0.201	783	1441

Decode speed and MTP self-speculative decoding (MMLU-Pro) — 24 questions × 1 seed, llama.cpp on one H200, 4 parallel slots

config	median tokens	tok/s	s / task	MTP speedup	accept_len (max 4)
f16 · standard	195	29.9	5.5	1.00×	—
f16 · MTP	213	52.7	3.7	1.76×	2.44
Q8_0 · standard	330	43.6	8.9	1.00×	—
Q8_0 · MTP	217	46.6	5.5	1.07×	2.52
Q6_K · standard	162	36.5	4.1	1.00×	—
Q6_K · MTP	224	48.9	6.3	1.34×	2.36
Q4_K_M · standard	241	37.5	6.4	1.00×	—
Q4_K_M · MTP	180	48.9	4.3	1.30×	2.33
IQ4_XS · standard	190	44.8	4.6	1.00×	—
IQ4_XS · MTP	243	54.1	4.0	1.21×	2.41

Where to find us

Need even more efficiency? The open release is production-ready. Our enterprise versions go further — fewer thinking tokens still, tuned to your workload, at matched accuracy on your own tasks. Built for AI labs, inference providers and enterprises running models at scale. Deployed on your infrastructure, or in the cloud and region you choose. Talk to our team

License

ThinkingCap: PolyForm Small Business 1.0.0 + BottleCap personal-use grant (see LICENSE).

Upstream Qwen materials: Apache-2.0 (see NOTICE).

Commercial license: contact BottleCap AI.

Citation

If you use this model, please cite:

@misc{ThinkingCap-Qwen3.8-27B,
  title     = {bottlecapai/ThinkingCap-Qwen3.8-27B},
  author    = {Osusky, Adam and Lindauer, Jan and Jirkovsky, Adam and Mihal, Filip and Platek, Ondrej and Herel, David and Ihnatchenko, Luka and Bartek, Vojtech and Jirak, Jiri and Kubista, Daniel and Krus, Frantisek and Mikolov, Tomas},
  year      = {2026},
}