AtomicChat/Clover-35B-A3B-Preview-GGUF

Clover-35B-A3B-Preview · GGUF

GGUF quants of DiffuSpace's Clover-35B-A3B-Preview, built from DiffuSpace's original BF16 weights on release day.

• Clover-35B-A3B-Preview is DiffuSpace's block-diffusion language model: a 35B mixture-of-experts with about 3B active parameters per token, derived from Qwen3.6-35B-A3B, 128K context, thinking on by default, Apache-2.0.
• Built by the team behind Atomic Chat, the app that runs open-weight models locally and offline.
• Four plain llama.cpp files from 21.2 GB to 36.9 GB on disk, each measured against the BF16 original on one stand.

---

These files need llama.cpp built from PR #31306 (commit 4838ffb, not merged yet). Released llama.cpp builds, Ollama, LM Studio and apps on stock llama.cpp do not know the clover_moe architecture yet. If the PR changes the file format before it is merged, we will re-quantize.

Built on release day, 28 October, with llama.cpp's own presets and no importance matrix.

Pick a file

KL divergence is how far the quantized model's predictions drift from the original BF16 weights. Lower is better, and zero means identical. top-1 is how often it picks the same next token the original would have picked. Every file was measured on the same stand: our neutral multilingual text, 87 chunks of 4,096 tokens, llama.cpp 4838ffb (PR #31306), plain decoding.

File	Size on disk	KL divergence	top-1	Perplexity vs BF16
Clover-35B-A3B-Preview-Q8_0.gguf	36.9 GB	0.0268	92.96%	+0.13%
Clover-35B-A3B-Preview-Q6_K.gguf	28.5 GB	0.0380	91.34%	+0.23%
Clover-35B-A3B-Preview-Q5_K_M.gguf	24.7 GB	0.0689	88.70%	+1.0%
Clover-35B-A3B-Preview-Q4_K_M.gguf	21.2 GB	0.1111	85.29%	+4.8%

Read these numbers against this model's noise floor. In this build, Clover's predictions move even when the weights do not: the BF16 file itself, run with only the micro-batch changed (128 instead of 512 tokens), has KL divergence 0.014 and top-1 94.9% against the same BF16 run (first 8 chunks); with only the K/V cache in bf16 instead of f16, 0.013 and 95.1% (first 10 chunks). For most models both are near zero. Perplexity barely moves (+0.13% for Q8_0): the quantized files swap near-equal candidates more often than they lose quality. Compare the files with each other.

Pick the largest file that fits: Q8_0 (36.9 GB) or Q6_K (28.5 GB), which stays close to it. Q5_K_M (24.7 GB) and Q4_K_M (21.2 GB) drift more (top-1 88.7% and 85.3%, perplexity +1.0% and +4.8%); take them when nothing larger fits, for example on a 24 GB card. Raw logs and the noise-floor runs are in the metrics repo.

How Clover runs in llama.cpp

Clover is trained to fill a block of masked tokens at once. llama.cpp uses that in two ways:

• Plain decoding (llama-server, llama-cli): one token at a time, like any model. Every number in the table above is measured this way.
• Self-speculative decoding (--spec-type draft-self, from the PR): the model drafts a masked block and checks it causally, with no separate draft model.

With these files draft-self works: on a code prompt every drafted token was accepted (101 of 101 on BF16, 108 of 108 on Q8_0, 100 of 100 on Q4_K_M). It is not fast yet in this build: on one A100-80GB it decoded 0.6-0.7 tokens/s, against about 53 tokens/s for plain decoding of Q4_K_M in llama-server, because the drafted blocks run slowly. The PR's author reports about 32 tokens/s on an H200 and plans speedups in later PRs. llama-server --spec-type draft-self does not start the self-speculative path in this build (it logs failed to initialize speculative decoding context and decodes plainly). For now, use plain decoding.

Run

Build llama.cpp from the PR:

git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
git fetch origin pull/31306/head && git checkout 4838ffb0c8ade92f34d9157deea38d92758ab824
cmake -B build -DGGML_CUDA=ON && cmake --build build -j --target llama-server llama-speculative-simple

Chat (plain decoding):

./build/bin/llama-server -hf AtomicChat/Clover-35B-A3B-Preview-GGUF:Q6_K --jinja -c 32768 \
    --presence-penalty 0.9 --repeat-penalty 1.05

Thinking is on by default. To turn it off: --chat-template-kwargs '{"enable_thinking": false}'. DiffuSpace's defaults are greedy decoding with presence penalty 0.9 and repetition penalty 1.05; their length and end-of-thinking controls (dllm_length_penalty, dllm_think_end_bias) belong to the diffusion sampler and are not in llama.cpp.

Self-speculative decoding (the PR's example program):

./build/bin/llama-speculative-simple -m Clover-35B-A3B-Preview-Q6_K.gguf --spec-type draft-self \
    --spec-draft-n-max 15 -ctk bf16 -ctv bf16 -fa on -ngl 99 -n 512 -p "..."

License and credits

The weights are DiffuSpace's, released under Apache-2.0 (LICENSE and ATTRIBUTIONS.md in this repo). Clover is derived from Qwen3.6-35B-A3B (Copyright 2026 Alibaba Cloud, Apache-2.0). Quantized by AtomicChat.