Instructions to use YTan2000/Qwen3.8-27B-TQ with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use YTan2000/Qwen3.8-27B-TQ with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf YTan2000/Qwen3.8-27B-TQ
# Run inference directly in the terminal:
llama cli -hf YTan2000/Qwen3.8-27B-TQ

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf YTan2000/Qwen3.8-27B-TQ
# Run inference directly in the terminal:
llama cli -hf YTan2000/Qwen3.8-27B-TQ

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf YTan2000/Qwen3.8-27B-TQ
# Run inference directly in the terminal:
./llama-cli -hf YTan2000/Qwen3.8-27B-TQ

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf YTan2000/Qwen3.8-27B-TQ
# Run inference directly in the terminal:
./build/bin/llama-cli -hf YTan2000/Qwen3.8-27B-TQ

Use Docker

docker model run hf.co/YTan2000/Qwen3.8-27B-TQ

LM Studio
Jan

vLLM

How to use YTan2000/Qwen3.8-27B-TQ with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "YTan2000/Qwen3.8-27B-TQ"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "YTan2000/Qwen3.8-27B-TQ",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/YTan2000/Qwen3.8-27B-TQ

Ollama
How to use YTan2000/Qwen3.8-27B-TQ with Ollama:
```
ollama run hf.co/YTan2000/Qwen3.8-27B-TQ
```
Unsloth Desktop

How to use YTan2000/Qwen3.8-27B-TQ with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf YTan2000/Qwen3.8-27B-TQ

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "YTan2000/Qwen3.8-27B-TQ"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use YTan2000/Qwen3.8-27B-TQ with Docker Model Runner:
```
docker model run hf.co/YTan2000/Qwen3.8-27B-TQ
```

Lemonade

How to use YTan2000/Qwen3.8-27B-TQ with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull YTan2000/Qwen3.8-27B-TQ

Run and chat with the model

lemonade run user.Qwen3.8-27B-TQ-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use YTan2000/Qwen3.8-27B-TQ with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf YTan2000/Qwen3.8-27B-TQ

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default YTan2000/Qwen3.8-27B-TQ

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use YTan2000/Qwen3.8-27B-TQ with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf YTan2000/Qwen3.8-27B-TQ

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "YTan2000/Qwen3.8-27B-TQ" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.8-27B-TQ

Engine: tqLLM 1.0 (docker pull ytan2000/tqllm:1.0.0 or ghcr.io/turbo-tan/tqllm:1.0.0).

Qwen3.8-27B for the tqLLM engine in four sizes (11.85 to 14.72 GB), with a DFlash2 draft model for exact speculative decoding. TQ_XS and TQ_S target 16 GB GPUs; TQ_L targets 24 GB GPUs and GB10.

Credits

What is in these files	Credit
Qwen3.8-27B weights, MTP head, chat template	Qwen team, Alibaba (Qwen)
DFlash2 draft model weights and method	Jian Chen, Yesheng Liang, Zhijian Liu, z-lab (arXiv 2602.06036)
BF16 source GGUF	unsloth/Qwen3.8-27B-GGUF
Fixed chat template (embedded)	froggeric, Qwen-Fixed-Chat-Templates
Rotation-based quantisation, the lineage of the TQ3 / TQ4 formats	Google Research, TurboQuant

Credits for the engine (KV cache, speculative decoding, kernels): tqllm.com/credits.

Files

File	Size	What it is
`Qwen3.8-27B-TQ_XS.gguf`	11.85 GB	smallest; 16 GB GPUs with the most room for context
`Qwen3.8-27B-TQ_S.gguf`	12.55 GB	16 GB GPUs; closer to BF16 than TQ_XS
`Qwen3.8-27B-TQ_M.gguf`	13.14 GB	experimental TQ3 / TQ4 mix; not in the quality-gated set (KLD only, no task scores)
`Qwen3.8-27B-TQ_L.gguf`	14.72 GB	24 GB GPUs and GB10; closest to BF16 (TQ4 tensors)
`drafters/Qwen3.8-27B-DFlash2-TQ3.gguf`	0.79 GB	DFlash2 draft model, TQ3 weights
`mtp-vocab/atx_65536.txt`	0.4 MB	the draft model's token list (65,536 ids)
`drafters/NOTICE`		licence notice for the draft model

Pick one model file (TQ_S shown) and download it with the drafter and its token list:

hf download YTan2000/Qwen3.8-27B-TQ Qwen3.8-27B-TQ_S.gguf drafters/Qwen3.8-27B-DFlash2-TQ3.gguf mtp-vocab/atx_65536.txt --local-dir models

For another size, replace Qwen3.8-27B-TQ_S.gguf with Qwen3.8-27B-TQ_XS.gguf, Qwen3.8-27B-TQ_M.gguf or Qwen3.8-27B-TQ_L.gguf (also in MODEL_FILE below).

Run

These files need tqLLM: their TQ3 / TQ4 tensors use tqLLM's own GGUF types (300, 301), so stock llama.cpp, Ollama and LM Studio cannot load them.

docker run --rm --gpus all -p 127.0.0.1:8086:8086 -v "$PWD/models:/models:ro" \
  -e MODEL_FILE=/models/Qwen3.8-27B-TQ_S.gguf \
  -e DRAFT_MODEL_FILE=/models/drafters/Qwen3.8-27B-DFlash2-TQ3.gguf \
  -e DRAFT_VOCAB_FILE=/models/mtp-vocab/atx_65536.txt \
  -e KV_FORMAT=tq3 -e CONTEXT=32768 -e REASONING=on -e MAX_TOKENS=4096 \
  ytan2000/tqllm:1.0.0

The image runs on x86-64 (RTX 30 / 40 / 50) and arm64 (GB10 / DGX Spark). The server is OpenAI-compatible (/v1/chat/completions). All options: docker run --rm --gpus all --entrypoint tqserve ytan2000/tqllm:1.0.0 --help.

Drafter: the draft model proposes several tokens and the 27B checks them in one pass; every accepted token is exact, so the output is the model's own text, only faster.
Long context: KV_FORMAT=tq3 stores the KV cache at about 3 bits per value. On 16 GB cards TqCache keeps the oldest KV in system RAM when the requested context does not fit on the GPU (exact, at a decode cost from PCIe).

Quality

	TQ_XS	TQ_S	TQ_M	TQ_L
Size	11.85 GB	12.55 GB	13.14 GB	14.72 GB
KLD vs BF16 (mean per-token KL divergence, lower is closer)	0.079	0.055	0.045	0.022
Same top token as BF16	88.6 %	90.1 %	91.2 %	93.7 %
GSM8K (1319, greedy, thinking on)	95.4 % *	94.6 %	-	94.9 %
MMLU-Pro (291-question subset)	78.7 % *	83.2 %	-	83.5 %
GPQA Diamond (198)	84.85 %	82.83 %	-	87.88 %

KLD on GB10. GSM8K / MMLU-Pro on GB10, except * on RTX 3090; GPQA on RTX 3090. tqLLM release engine, DFlash2-TQ3 drafter, tq3 KV. MMLU-Pro on 291 questions has a standard error of about 2.4 points; GSM8K differences between the files are within noise. TQ_M has KLD only.

BenchLoop by task (RTX 3090, same engine):

Task	TQ_XS	TQ_S	TQ_L
speed	92.6	93.2	91.9
toolcall	86.7	86.7	86.7
coding	79.2	85.4	87.5
dataextract	92.9	89.0	88.4
instructfollow	76.7	91.1	84.5
reasonmath	80.0	80.0	80.0

Speed (LocalMaxxing canonical prompts, DFlash2-TQ3 drafter, 32K context, thinking on, median tok/s)

Hardware	TQ_XS reasoning / code	TQ_S reasoning / code	TQ_L reasoning / code
RTX 3090 24 GB	271.3 / 149.8	251.7 / 173.4	-
RTX 5060 Ti 16 GB (Windows 11 / WSL 2)	175.9 / 98.8	166.5 / 109.0	-
GB10 (DGX Spark)	99.7 / 67.6	92.8 / 60.4	72.4 / 61.0

All rows use the drafter (speculative decoding, exact output). Median of 5 runs (RTX 3090: 3 runs). "-" = not measured yet. Decode speed with a drafter depends on the text: code and copy-heavy answers accept more drafts than open prose.

Format

TQ3 (3.125 bits per weight) and TQ4 (4.125) rotate each 128-weight block and store one FP16 scale per block plus 8- or 16-level codes; the rotation spreads outliers so one scale per 128 weights is enough. For comparison, TQ3_4S (the format of YTan2000/Qwen3.8-27B-TQ3_4S) uses 32-weight blocks with four 8-bit scales, 4.0 bits per weight.