Instructions to use 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF",
	filename="Qwen3.6-27B-Q8_0_ROCMFPX.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX
# Run inference directly in the terminal:
llama cli -hf 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX
# Run inference directly in the terminal:
llama cli -hf 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX
# Run inference directly in the terminal:
./llama-cli -hf 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX
# Run inference directly in the terminal:
./build/bin/llama-cli -hf 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX

Use Docker

docker model run hf.co/1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX

LM Studio
Jan

vLLM

How to use 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX

Ollama

How to use 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF with Ollama:

ollama run hf.co/1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX

Unsloth Studio

How to use 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF to start chatting

How to use 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF with Docker Model Runner:
```
docker model run hf.co/1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX
```

Lemonade

How to use 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF:Q8_0_ROCMFPX

Run and chat with the model

lemonade run user.Qwen3.6-27B-Q8_0-ROCMFPX-GGUF-Q8_0_ROCMFPX

List all available models

lemonade list

Qwen3.6-27B — Q8_0_ROCMFPX GGUF (experimental, AMD gfx1201)

An experimental Q8 quantization of Qwen/Qwen3.6-27B, tuned and validated for the AMD Radeon AI PRO R9700 (gfx1201).

This file does not run on upstream llama.cpp, Ollama, LM Studio, or vLLM. It uses the custom Q8_0_ROCMFPX tensor type and requires the pinned ROCmFPX fork build described below. Unsupported runtimes should reject the file; if a tool appears to load it anyway, do not trust the output. Hugging Face's GGUF metadata viewer may also mislabel the custom tensor type or fail to parse the file.

Should you use this?

Use this model if all of the following are true:

you run a Radeon AI PRO R9700 (gfx1201) under ROCm;
you are willing to build the pinned fork from source;
you want a Q8 that is 2.94% smaller than upstream Q8_0 and performs equal observed model work about 6% faster on this hardware.

Otherwise, use an ordinary Q8_0 GGUF of Qwen3.6-27B. It has broad support in the standard GGUF runtime ecosystem, and its quality is equivalent within the tolerances measured here.

TL;DR — measured against upstream `Q8_0`

Measure	Result
Model size	2.94% smaller
HumanEval	140/164 — tied
MBPP base / MBPP+	348/378 and 291/378 vs 350/378 and 293/378 — within the predeclared tolerance
Proxy — full-model decode	5.77–5.87% faster
Proxy — `pp4096+tg512` combined	4.75–4.96% faster
Equal-work — agent-derived evaluation (18 matched pairs)	5.82% faster median, 17/18 pairs, exact one-sided `p=0.000965`
End-to-end — raw live-agent wall time	2.21% faster — failed the predeclared 3% threshold

The supported conclusion is deliberately narrow: this format performs equal observed model work faster than upstream Q8_0 on the measured deployment. The campaign did not confirm that complete live-agent jobs finish at least 3% faster, so no such claim is made. All thresholds were fixed before the runs; failed gates are reported, not reinterpreted.

The public claim registry records the scope, result class, and sealed aggregate-evidence hash behind each release claim.

What `Q8_0_ROCMFPX` actually is

Each block stores 32 signed eight-bit codes and one finite UE4M3 scale byte: 33 bytes per 32 weights, or 8.25 bits per weight. The HIP path decodes the scale and performs integer MMVQ/MMQ dot products with float accumulation. This is not native FP8 or FP4 matrix arithmetic.

GGUF block
  32 signed int8 codes + UE4M3 scale
        |
        v
activation quantization to Q8_1
        |
        +--> decode:  gfx1201 MMVQ, VDR8 + measured wave policy
        |
        `--> prefill: integer MMQ path
        |
        v
float accumulation / model output

The experimental format and execution path come from the upstream ROCmFPX project. This repository's contribution is the gfx1201 decode tuning (VDR8 vector-dot width and a measured wave policy) and the validation below; other GPU targets retain ROCmFPX defaults and are unmeasured.

The quantization is uniform Q8 with no importance matrix and no per-tensor routing.

Required runtime

Build the pinned fork branch:

git clone https://github.com/1337hero/ROCmFPX.git
cd ROCmFPX
git checkout 45bcff509c4b1cff137e2cc1ea84671c61ceddea
env JOBS=16 scripts/build-r9700.sh

The wrapper is the simplest supported build. The sealed performance runners used this HIP-only configuration:

cmake -S . -B build-r9700 -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DGGML_HIP=ON \
  -DGGML_HIP_FORCE_MMQ=ON \
  -DGGML_VULKAN=OFF \
  -DGGML_CUDA=OFF \
  -DCMAKE_HIP_ARCHITECTURES=gfx1201 \
  -DGPU_TARGETS=gfx1201 \
  -DCMAKE_HIP_FLAGS= \
  -DLLAMA_BUILD_SERVER=ON \
  -DLLAMA_BUILD_WEBUI=OFF \
  -DLLAMA_USE_PREBUILT_WEBUI=OFF \
  -DLLAMA_BUILD_TESTS=ON \
  -DGGML_BUILD_TESTS=OFF
cmake --build build-r9700 -j 16 --target \
  llama-server llama-bench llama-quantize

The validated implementation is ROCmFPX base commit 6bf20cd688ba0af882d1f68ba50b292edf646ab4 plus commits eb38c6f67701ff9c74e8597f573eedf9ccecf774 and 45bcff509c4b1cff137e2cc1ea84671c61ceddea.

Binaries validated by the lab:

Binary	SHA-256
`llama-server`	`1471753a94ba007b094842474cc3b3ffd48106f15a7c71934899dd832ae4cbb3`
`llama-bench`	`38bc30e53badc5ae5efb5e3d449d989a4672c52fe5801eb1ce74e70233283749`
`llama-quantize`	`0aa545af25e8235613349987fb29323745b70980aa2a4fb45c169cdf561181c6`

Example deployment

This shape matches the validated two-card, 262,144-token envelope. Device names depend on the host; check --list-devices first.

./build-r9700/bin/llama-server \
  -m Qwen3.6-27B-Q8_0_ROCMFPX.gguf \
  --no-mmap \
  -c 262144 \
  -b 2048 \
  -ub 512 \
  -t 16 \
  -ngl 99 \
  -sm layer \
  -ts 1,1 \
  -dev ROCm0,ROCm2 \
  --fit off \
  -ctk f16 \
  -ctv f16 \
  -fa on \
  -np 1

Both this model and the upstream Q8_0 control allocated one 262,144-token slot across two R9700s with F16 K/V cache and generated bounded valid output; this model used 804,319,232 fewer resident bytes in the captured snapshots. That test proved allocation and bounded generation, not a timed full-context prompt.

Measurement conditions

All performance numbers were measured on:

Qwen3.6-27B, this GGUF vs an upstream Q8_0 control of the same conversion, same host, same workload;
two non-display Radeon AI PRO R9700s at a pre-existing 250 W, −75 mV tune — not stock 300 W;
ROCm 7.2.4 on Arch Linux, outside the official Radeon Linux support matrix;
pinned source revisions, forced model residency, reversed candidate order, and predeclared promotion thresholds.

The equal-work result comes from 18 common-seed matched agent-workload pairs with ABBA/BAAB counterbalancing, warmup before every measured run, full request accounting (1,899 logical requests, zero retries), and exact paired Wilcoxon signed-rank tests.

Artifact

Field	Value
File	`Qwen3.6-27B-Q8_0_ROCMFPX.gguf`
Size	27,755,345,216 bytes
SHA-256	`ff4dbc9093c1df6fd1242294d15eb94c7bfe42ed67f98e9d58b9789ac6912c1b`
Source GGUF	BF16, SHA-256 `0438be1f5bc861ffa84e1d2d4036920f6f3d9759f3cdedbc40e554a321d1c9c5`
Quantization	uniform `Q8_0_ROCMFPX`, no importance matrix
Content	main text model only; no MTP artifact or multimodal projector

Quantization command (the qualifying run wrote to a temporary filename and then promoted the verified hash without overwriting an existing artifact):

./build-r9700/bin/llama-quantize \
  Qwen3.6-27B-BF16.gguf \
  Qwen3.6-27B-Q8_0_ROCMFPX.gguf \
  Q8_0_ROCMFPX \
  16

Verify after download:

sha256sum -c SHA256SUMS

Limitations

Requires an experimental fork; upstream llama.cpp compatibility awaits a separate code contribution.
Validated on exactly one model conversion, one Arch Linux host, ROCm 7.2.4, and gfx1201 GPUs at a tuned power profile. Other GPUs, hosts, and stock power are unmeasured.
Aggregate quality stays within the declared tolerances, but this model and upstream Q8_0 do not produce identical per-task outcomes.
The raw private lab archive (prompts, outputs, paths, environment) is not published; it awaits a separate sanitization review.

License and attribution

Base model: Qwen3.6-27B, Copyright 2026 Alibaba Cloud, Apache-2.0. This repository redistributes a converted and quantized derivative under the same license. See LICENSE and NOTICE.
Format and execution path: the experimental Q8_0_ROCMFPX representation and HIP kernels are the work of the ROCmFPX project, which builds on llama.cpp.
This repository: the gfx1201 decode tuning, quantized artifact, and validation evidence.

Qwen and related marks belong to their owners. This community quantization is not affiliated with or endorsed by Qwen, Alibaba Cloud, AMD, ROCmFPX, or llama.cpp.

Downloads last month: 120

GGUF

Model size

27B params

Architecture

qwen35

Hardware compatibility

8-bit

Model tree for 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF

Base model

Qwen/Qwen3.6-27B

Quantized

(608)

this model

Qwen3.6-27B — Q8_0_ROCMFPX GGUF (experimental, AMD gfx1201)

Should you use this?

TL;DR — measured against upstream Q8_0

What Q8_0_ROCMFPX actually is

Required runtime

Example deployment

Measurement conditions

Artifact

Limitations

License and attribution

Model tree for 1337Hero/Qwen3.6-27B-Q8_0-ROCMFPX-GGUF

TL;DR — measured against upstream `Q8_0`

What `Q8_0_ROCMFPX` actually is