Harsha001/Clover-35B-A3B-Preview-ROCmFPX

Clover-35B-A3B-Preview — ROCmFPX GGUF

Diffuspace/Clover-35B-A3B-Preview is a block diffusion language model from DiffuSpace with 35B total parameters and about 3B active per token (mixture of experts, 256 experts, 8 per token, one shared expert). It starts from Qwen3.6-35B-A3B, is text only, and has a 128K context. This repo has it in the ROCmFPX formats, a llama.cpp fork with low-bit weight types for AMD GPUs (Vulkan and HIP/ROCm).

Start with Clover-35B-A3B-Preview-Q6_K_ROCMFPX.gguf (28.7 GB): it is almost as close to the full model as the 8-bit file and closer than a stock Q6_K of the same size. Take the 8-bit file if memory is no concern. For 4-bit, see the note under the tables: a stock Q4_K_M is the better choice for this model when quality matters.

These files need HarshaVardhan3002/ROCmFPX, built from 29 October 2026 or later (commit 642acd395; Clover support was added on 28 October). Stock llama.cpp, LM Studio, Ollama and other ROCmFPX builds may not load them. These files have not been tested on AMD hardware yet.

Not an official release of Clover. All credit for the model goes to DiffuSpace. Their own code, with SGLang and llama.cpp backends, is at DiffuSpace/Clover-35B-A3B-Preview.

Files

File	Size	KL to BF16	Same top token	PPL
Clover-35B-A3B-Preview-Q8_0_ROCMFPX.gguf	35.8 GB	0.0622	89.3%	9.49	8-bit
Clover-35B-A3B-Preview-Q6_K_ROCMFPX.gguf	28.7 GB	0.0652	88.9%	9.36	recommended, 6-bit (ROCmFPX Q6_0_ROCMFPX weights, see below)
Clover-35B-A3B-Preview-Q4_0_ROCMFP4_STRIX_O8.gguf	18.8 GB	0.1848	80.9%	9.92	4-bit, Strix Halo recipe, embeddings at 8-bit
Clover-35B-A3B-Preview-Q4_0_ROCMFP4_FAST.gguf	18.5 GB	0.1865	80.4%	9.69	smallest, speed-first 4-bit

The BF16 model scores PPL 9.45 on the same text. Stock llama.cpp files of this model, measured the same way:

Stock file	Size	KL to BF16	Same top token	PPL
Q8_0	36.9 GB	0.0539	89.7%	9.42
Q6_K	28.5 GB	0.0727	88.0%	9.67
Q4_K_M	21.2 GB	0.1191	84.3%	9.54

The 6-bit file here is closer to BF16 than a stock Q6_K at the same size. A stock Q8_0 is a little closer than the 8-bit file here, which is 1.1 GB smaller. At 4-bit, a stock Q4_K_M stays clearly closer to the full model than the ROCmFP4 files here, for 2.4 to 2.7 GB more. The ROCmFP4 files are smaller and are meant for speed on AMD GPUs; pick the stock Q4_K_M if quality matters more than speed.

Every quantized file of this model, stock or ROCmFPX, drifts further from BF16 than the same type does on other Qwen3.6-based models (for example a stock Q8_0 has KL 0.054 here and 0.008 on Groot-19B-A3B). This is the model, not noise in the test: the BF16 file measured against itself gives KL 0.000000 and the same top token 99.99% of the time. Perplexity and the math test below barely move, so the drift is spread over tokens that matter little.

How this was measured: llama.cpp llama-perplexity --kl-divergence against the BF16 conversion of the original weights, 40 chunks of 512 tokens of English Wikipedia text (wikitext-2), on the CPU. KL is how far the file's next-token probabilities drift from BF16, averaged over every token; lower is better. "Same top token" is how often the file's most likely next token is the same as BF16's.

Math test (GSM8K)

The first 100 questions of the GSM8K test set, thinking switched off (enable_thinking: false), greedy decoding, at most 1024 tokens per answer, the answer read from a last line "Final answer: ". The model ran as a normal next-token model in llama-server (one token at a time, no block diffusion).

File	Correct (of 100)
BF16	97
stock Q4_K_M	97
Q8_0_ROCMFPX	99
Q4_0_ROCMFP4_STRIX_O8	97
Q4_0_ROCMFP4_FAST	98

On 200 questions BF16 got 192, stock Q4_K_M 190 and the 8-bit file 195. With 100 questions a difference of one or two is noise, so none of the files lost measurable math ability. The 6-bit file answered 38 of its first 39 questions right before the run was stopped (it is very slow on the test machine's CPU, see below). All of these ran on the CPU.

Other checks

Every file answered "What is the capital of France?" correctly (greedy, CPU).

Speed

These files were made and tested on an ARM machine (NVIDIA DGX Spark CPU, 20 cores), not on an AMD GPU, so there are no AMD speed numbers yet. On that CPU the fork's code for the 6-bit and 4-bit ROCmFPX types is slow, which says nothing about their speed on AMD GPUs, where they run on different code:

File	Prompt (tok/s)	Generation (tok/s)
stock Q4_K_M	99.8	22.8
Q8_0_ROCMFPX	90.3	17.3
Q6_K_ROCMFPX	2.8	0.8
Q4_0_ROCMFP4_STRIX_O8	21.9	2.2
Q4_0_ROCMFP4_FAST	21.4	2.2

(llama-bench, 512-token prompt, 128 generated tokens, 20 threads.) If you run on a CPU, use the 8-bit file or a stock file.

Notes on the files

All four files carry per-tensor scales: each weight matrix is multiplied by a power of two before quantizing and the inverse is stored as <name>.scale, which the fork multiplies back after the matrix product. This keeps small weights from getting stuck at the smallest ROCmFPX block scale. The token embedding uses a stock llama.cpp type (Q8_0 in the 8-bit and STRIX_O8 files, Q6_K in the 6-bit file; the FAST file keeps it in ROCmFP4 for speed).

The 6-bit file is named ...-Q6_K_ROCMFPX.gguf so that Hugging Face lists it as a 6-bit quant. Its weights are the ROCmFPX Q6_0_ROCMFPX type, not a stock llama.cpp Q6_K, and it needs the same fork as the other files.

The files keep Clover's extra block attention weights (the right convolution and softmax controls of the Gated DeltaNet layers) and its expert-selection settings, so the fork's masked-block self-speculative decoding (llama-speculative-simple --spec-type draft-self, see examples/speculative-simple/README.md in the fork) can use them. That mode was not tested with these files.

How these were made

1. Source: the original BF16 safetensors from Diffuspace/Clover-35B-A3B-Preview, converted to a BF16 GGUF with convert_hf_to_gguf.py from the fork.
2. Importance matrix from the BF16 file over a general calibration text (calibration_datav3), 512-token chunks.
3. llama-quantize --imatrix from the BF16 file with LLAMA_ROCMFPX_TENSOR_SCALE=1, one run per type, from HarshaVardhan3002/ROCmFPX.

The only change from the original is the weight quantization. The tokenizer and chat template come unchanged from the original model.

Run

git clone https://github.com/HarshaVardhan3002/ROCmFPX && cd ROCmFPX
cmake -B build -DGGML_VULKAN=ON          # or -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 (gfx1100 for RX 7900)
cmake --build build --target llama-server -j
build/bin/llama-server -m Clover-35B-A3B-Preview-Q6_K_ROCMFPX.gguf -ngl 999 -c 131072 --jinja \
  --presence-penalty 0.9 --repeat-penalty 1.05

The model thinks before it answers by default. DiffuSpace's defaults are greedy decoding with presence penalty 0.9 and repetition penalty 1.05; they note that very long generations can become repetitive. To answer without thinking, send "chat_template_kwargs": {"enable_thinking": false} with the request. DiffuSpace recommend their own SGLang implementation for serving and evaluation.

License

Apache-2.0, the same as the original model (derived from Qwen3.6-35B-A3B, Copyright 2026 Alibaba Cloud).