Qwen3.6-35B-A3B-8bit-paged

Qwen3.6-35B-A3B, quantized to 8 bits and packaged so that a machine which cannot hold the whole model in memory can still run it.

The model is Qwen/Qwen3.6-35B-A3B. This repository changes only how the weights are laid out on disk, not what they are.

Output is bit-identical to the same weights in their ordinary layout — the same bits, not merely close. The comparison is made while the repository is built, when both layouts still exist, and its result is recorded inside the repository.

How it works

A mixture-of-experts model only uses a few experts for any given token, but a normal checkpoint keeps all of them in memory all the time. Here the expert weights are stored separately, in a layout that allows any single expert to be read on its own.

At load time the engine looks at how much memory the machine actually has:

• Enough for the whole model — the experts are read into memory and the model runs exactly as the original does. Nothing is streamed and nothing is different.
• Not enough — the experts stay on disk, and each layer keeps a bounded cache of the ones it is using. The rest are read as the routing asks for them.

There is nothing to configure and no separate variant to choose. One repository serves both cases.

The repository is no larger for it: the expert tensors were moved out of the safetensors files, not duplicated.

What it costs

Streaming experts from disk is slower than having them in memory. If your machine has room for the model you pay nothing at all; if it does not, this is the trade that makes the model runnable rather than impossible.

Processing a long prompt for the first time is where the difference is most visible. Continued conversations are much less affected, because the prompt cache reuses work already done.

Requirements

gbx-lm with expert paging support. Earlier versions do not recognise this format — the expert weights are not where they expect to find them.

from gbx_lm.utils import load
model, tokenizer = load("GreenBitAI/Qwen3.6-35B-A3B-8bit-paged")

python -m gbx_lm --model GreenBitAI/Qwen3.6-35B-A3B-8bit-paged

GBX_PAGING=off disables streaming entirely, for a machine that has the memory and wants to be certain it is not being used.

Limits

One request at a time. The expert cache is per-model state, so concurrent requests would interfere with each other. The engine detects this and serves without streaming instead — which means a machine that depends on streaming cannot serve concurrent requests.

Files

config.json, tokenizer*     as published upstream, with two additions
model.safetensors           every weight that is not an expert
experts.bin                 the expert weights
experts_index.json          their layout, and the build-time verification record

The two additions to config.json tell the engine which weight format these are and how the expert projections are quantized — the latter would otherwise be inferred from tensors that now live in experts.bin.