Qwen3.5-2B · Glyd swift
1.3 GB, 65% smaller than bf16 (3.8 GB). Glyd's swift level of Qwen/Qwen3.5-2B (commit 15852e8c): about 5.5 bits per weight, quantized once from the original bf16 weights. Not lossless: KL divergence from bf16 is 0.0224 (lower is closer).
curl -LsSf https://getglyd.com/install.sh | sh
glyd run Qwen/Qwen3.5-2B:swift
Compared
| Size | Smaller than bf16 | KL vs bf16 | RTX 4090 | |
|---|---|---|---|---|
| bf16 (original) | 3.8 GB | – | 0 | – |
| kestrel | 1.5 GB | 59% | 0.00581 | 396 tok/s |
| swift (this repo) | 1.3 GB | 65% | 0.0224 | 443 tok/s |
Size: the files you download. KL: how far the model's next-token probabilities move from bf16, measured on WikiText-2.
Speed and memory
| GPU | Tokens/s | First token | Memory at 4k | 32k context |
|---|---|---|---|---|
| RTX 4090 | 443 | 47 ms | 2.2 GB | yes |
| L40S | 355 | 41 ms | 2.3 GB | yes |
| L40 | 352 | 45 ms | 2.3 GB | yes |
| RTX A6000 | 295 | 71 ms | 2.0 GB | yes |
Measured 2026-10-09 with glyd 0.29.4: one GPU each, a 4k-token context; memory is the GPU memory in use.
Fits a 32k context on: RTX 4090, L40S, L40, RTX A6000.
Good to know
- Runs on Glyd's engine (
glyd run): Linux with an NVIDIA GPU, driver 580 or newer. Not for vLLM or transformers yet. - Text only: the model's vision part is not included.
- The Hub's "8-bit precision" and "Model size" labels count packed bytes: this checkpoint is about 5.5 bits per weight, from a model with 1,881,825,088 parameters.
License
Weights: Apache-2.0, from Qwen/Qwen3.5-2B. Glyd's engine, which runs them: BUSL-1.1, free for personal and non-commercial use on your own computers; commercial use needs a license. getglyd.com
- Downloads last month
- 53