glyd/Qwen3.5-9B-swift

Qwen3.5-9B · Glyd swift

6.2 GB, 65% smaller than bf16 (17.9 GB). Glyd's swift level of Qwen/Qwen3.5-9B (commit c2022362): about 5.5 bits per weight, quantized once from the original bf16 weights. Not lossless: KL divergence from bf16 is 0.0373 (lower is closer).

curl -LsSf https://getglyd.com/install.sh | sh
glyd run Qwen/Qwen3.5-9B:swift

Compared

	Size	Smaller than bf16	KL vs bf16	RTX 4090
bf16 (original)	17.9 GB	–	0	–
penguin (lossless)	12.7 GB	29%	0	–
kestrel	7.3 GB	59%	0.0147	122 tok/s
swift (this repo)	6.2 GB	65%	0.0373	138 tok/s

Size: the files you download. KL: how far the model's next-token probabilities move from bf16, measured on WikiText-2.

Speed and memory

GPU	Tokens/s	First token	Memory at 4k	32k context
RTX 4090	138	140 ms	7.5 GB	yes
L40S	108	128 ms	7.6 GB	yes
RTX A6000	102	213 ms	7.4 GB	yes
RTX 2000 Ada Generation	35	507 ms	7.2 GB	yes

Measured 2026-10-10 with glyd 0.29.4: one GPU each, a 4k-token context; memory is the GPU memory in use.

Fits a 32k context on: RTX 4090, L40S, RTX A6000, RTX 2000 Ada Generation.

Benchmarks

Benchmark	this level	bf16
GSM8K	94.8%	95.5%

Run through glyd serve with Inspect AI 0.3.278 (inspect_evals 0.24.0); the bf16 column is the source model in bf16 served by vLLM, with the same harness and settings. GSM8K: 10-shot, numeric match, non-thinking, 1319 questions.

Good to know

• Runs on Glyd's engine (glyd run): Linux with an NVIDIA GPU, driver 580 or newer. Not for vLLM or transformers yet.
• Text only: the model's vision part is not included.
• The Hub's "8-bit precision" and "Model size" labels count packed bytes: this checkpoint is about 5.5 bits per weight, from a model with 8,953,803,264 parameters.

License

Weights: Apache-2.0, from Qwen/Qwen3.5-9B. Glyd's engine, which runs them: BUSL-1.1, free for personal and non-commercial use on your own computers; commercial use needs a license. getglyd.com