SENTYMENTINEL/wpc-qwen3-coder-30b-a3b-4bit

WPC v4 — Qwen3-Coder-30B-A3B at 4.25 bits per weight

A 30-billion-parameter mixture-of-experts coding model compressed to 15.10 GB and running on a 2016 quad-core laptop CPU with no GPU, at conversational-adjacent speed.

Compressed with wpc-engine — a tensor compilation and inference engine written in pure Rust, with no Python runtime and no external inference dependencies.

Summary

	Source (bf16)	WPC v3	WPC v4 (this repo)
Bits per weight	16	6.25	4.25
Size on disk	57.0 GB	22.21 GB	15.10 GB (15 462 MiB)
Decode throughput	—	1.00 tok/s	2.35 tok/s
Words per minute	—	~51	~120

Throughput measured on an Intel Core i7-6820HQ (Skylake-H, 4 cores / 8 threads, 2.70 GHz base, AVX2 + FMA, DDR4 dual-channel), greedy decoding, batch size 1, no GPU. Peak 2.33 tok/s, 2.35 tok/s in steady state.

The 2.35x speed-up over v3 comes from two independent changes: the narrower 4.25-bit encoding, and a tensor ordering change that stores each expert as one contiguous ~3.5 MB run instead of scattering its three projections across the file. Measured in isolation on v3, the ordering change alone is worth +45% (1.16 → 1.68 tok/s). Weight values are untouched by it — reconstruction is bit-identical.

Format

Each block of 128 consecutive weights is stored in 68 bytes:

Field	Size
zero_point	fp16, 2 bytes
scale	fp16, 2 bytes
128 × 4-bit codes, two per byte	64 bytes

68 bytes / 128 weights = 4.25 bits per weight. Reconstruction is w = zero_point + code * scale.

Routers are deliberately left uncompressed. Expert selection is a discrete argmax over 128 logits, where quantisation error changes which experts run rather than merely by how much.

Quality

Evaluated on 12 tasks spanning code synthesis, general knowledge, arithmetic, reasoning and tool use.

Tool calls: 3/3 correct, with no extra prose and correct end-of-turn termination:

• read_file('README.md')
• search_web(query="current price of bitcoin")
• list_files(src) emitted as the first step of a two-step plan

Code synthesis is unaffected. On a dense control model (Qwen3-4B) at the same bit width, the 4.25-bit build produced token ids identical to the 6.25-bit build on a 40-token code completion — the compression is not merely "good enough" there, it is indistinguishable.

Known weakness — translation into Polish. In one translation task the model used pojutrze ("the day after tomorrow") where the source said tomorrow, and appended a sentence that was not in the source. Non-English generation is the weakest observed area and is not recommended for unsupervised use.

Why 4.25 bits and not less

A 2.25-bit variant (v5, 36 bytes per 128 weights) was built and measured. It is smaller but not faster, and its output is gibberish:

Scheme (Qwen3-4B, dense)	Size	40 tokens	Output
v3, 6.25 bits	3.0 GB	18.03 s	correct code
v4, 4.25 bits	2 038 MiB	11.84 s	correct code, token ids identical to v3
v5, 2.25 bits	1 079 MiB	12.40 s	degenerate

Below roughly 2 GB the model stops being limited by memory bandwidth, so further compression buys no speed while it does cost quality. 4 bits is an optimum, not a compromise.

Files

File	Contents
model_v4.wpc	Compressed weights, memory-mapped at load (load time under 0.5 s)
model_v4.meta	Tensor index: names, shapes, offsets
runtime/	Tokenizer, config.json, generation config and uncompressed norms

Usage

Build the engine from the repository, then:

wpc-runtime --model <runtime dir> \
            --wpc <dir containing model_v4.wpc> \
            --scheme v4 \
            --prompt "def binary_search(arr, target):" \
            --max-tokens 60

Requires a CPU with AVX2 and FMA. Approximately 16 GB of free RAM is recommended; the artifact is memory-mapped rather than copied, so it also runs with less at the cost of page-cache misses.

Known limitation

The engine currently processes one token at a time (forward(token)). There is no batching, so reading a prompt costs the same per token as writing a reply — for example, prefill (28 tokens) in 65.2s. Long prompts are expensive, and speculative decoding and expert-grouped execution are blocked until batched forward passes land. This is the principal outstanding item of work.

Licence and a request

Apache 2.0, inherited from the base model.

Free and Open Source. However, if you monetize this project, you are kindly asked to donate 1% of your profits to a charity supporting neurodivergent individuals, honoring the project author's request.

This is a request, not a licence condition. It is made in good faith and left to yours.

Full technical report

Measurements, parameter accounting, the tensor-ordering study, the bit-width scaling study and a post-mortem of the abandoned v1 codebook scheme are in WHITEPAPER.md.