brasa-v0.01-base
Quantized weights of Qwen3-4B in the native .brasa
format of Brasa, a local inference engine for Apple Silicon
written in Rust on Metal. Brasa serves an OpenAI- and Anthropic-compatible API on your Mac and is
built mainly to run coding agents (Claude Code, Codex, Cline, OpenCode) against a local model.
These files only work with Brasa. They are not GGUF, MLX or safetensors.
Variants
Each variant is a folder that Brasa loads as one model (model.brasa + the Qwen3 tokenizer):
| Folder | Linear layers | Embeddings / lm_head (tied) |
Norms | model.brasa |
|---|---|---|---|---|
qwen3-4b-q4 (default) |
q4_0, groups of 32 |
q6_0 |
f32 | 2.36 GB |
qwen3-4b-q4-e8 |
q4_0, groups of 32 |
q8_0 |
f32 | 2.46 GB |
q4_0: blocks of 32 weights along the input dimension, an f16 scale and 16 bytes of nibbles (4.5 bits per weight),w = d · (q − 8).q6_0: blocks of 32, an f16 scale and 6-bit values (6.5 bits per weight),w = d · (q − 32).q8_0: blocks of 32, an f16 scale and 32 × i8 (8.5 bits per weight).
qwen3-4b-q4 is what Brasa uses by default: its q6_0 table reads 97 MB less per decoded token
than q8_0. qwen3-4b-q4-e8 keeps the earlier q8_0 table for comparison.
Quality
Measured with teacher forcing over the 2898 positions of Brasa's reference fixtures, against the unquantized FP32 model (Brasa ADR 0012):
| Embedding table | Top-1 agreement with FP32 | Mean top-20 KL | p99 KL |
|---|---|---|---|
q8_0 (qwen3-4b-q4-e8) |
88.58 % | 9.42e-2 | 1.11 |
q6_0 (qwen3-4b-q4) |
88.34 % | 9.49e-2 | 1.06 |
Brasa's Metal kernels are checked against a CPU reference with these weights (tolerances in the repository's ADRs 0006, 0009 and 0012).
Speed and memory
Measured with brasa benchmark on an M1 Pro 16 GB (macOS 27.0), qwen3-4b-q4, one request,
against llama.cpp (Q4_0 GGUF of the same model) in the same harness. The reports, with chip, RAM,
macOS and commits, are in the repository's
docs/bench/baseline.md.
| Context | Prefill tok/s, Brasa / llama.cpp | Decode tok/s, Brasa / llama.cpp | Memory GiB, Brasa / llama.cpp |
|---|---|---|---|
| 2048 | 299 / 490 | 50.2 / 50.8 | 2.55 / 3.18 |
| 8192 | 195 / 356 | 34.6 / 38.0 | 3.40 / 4.03 |
| 16384 | 138 / 260 | 28.0 / 28.6 | 4.54 / 5.20 |
| 16384, KV Q8 | 134 / 260 | 27.1 / 19.7 | 3.48 / 4.13 |
Prefill is still slower than llama.cpp's; that is the current work in progress. Other Macs have not been measured.
Usage
Brasa downloads these files itself and checks each one's sha256:
brasa pull qwen3-4b-q4 # or qwen3-4b-q4-e8
brasa serve qwen3-4b-q4 --ctx 16384
# OpenAI: http://127.0.0.1:8080/v1/chat/completions and /v1/responses
# Anthropic: http://127.0.0.1:8080/v1/messages
# GUI: http://127.0.0.1:8080/ui
The Models screen of the GUI can also download a variant, and lets you choose the folder where models are stored.
Format
A .brasa file starts with a 16-byte header (BRSA, version, JSON length) and a JSON directory
with the model config, the source repository and commit, and every tensor's dtype, shape, offset
and sha256. The tensor data follows, each tensor aligned to 16 KiB so Brasa can mmap the file and
wrap each tensor in a Metal buffer without copying. brasa models verify <model> recomputes every
hash.
Source and license
- Base model: Qwen/Qwen3-4B, revision
1cfa9a7208912126459214e8b04321603b3df60c, converted from the BF16 safetensors with Brasa's converter. There is no fine-tuning: the weights are only quantized. - License: Apache-2.0, the same as Qwen3-4B (see
LICENSE). Qwen3 is developed by the Qwen team at Alibaba Cloud. - The answers are the model's: quantization lowers quality slightly (see above), and Qwen3-4B can be wrong or write unsafe code like any small model.