IsValorum/llama.cpp-clover-moe

llama.cpp with Clover support (clover_moe)

Prebuilt Clover runtime by IsValorum

Independently computed using dedicated cloud compute clusters, funded 100% out-of-pocket.

If you consider these releases to be of high quality and they help you in your day-to-day workflow, consider supporting on Ko-fi. Every contribution directly sustains future cloud compute hours and helps support my university studies.

Prebuilt Linux and Windows x86_64 binaries for Diffuspace/Clover-35B-A3B-Preview, built from llama.cpp PR #31306 at commit 4838ffb0c8ade92f34d9157deea38d92758ab824.

Mainline llama.cpp does not load clover_moe. This build adds the converter, the runtime graph and the draft-self speculative mode. The PR is not merged upstream; pin this repository for reproducibility.

Clover artifacts for this runtime

Artifact	Size	Link
BF16 source	69.38 GB	Clover-35B-A3B-Preview-BF16-GGUF
Q8_0	36.91 GB	Clover-35B-A3B-Preview-Q8_0-GGUF
1.25M I-Matrix	383,689,280 bytes	Clover-35B-A3B-Preview-VAL-1.25M-I-Matrix-GGUF
APEX-I NanoPlus V2.1	12.56 GB / 11.70 GiB	Clover-35B-A3B-Preview-APEX-I-NanoPlus-V2.1-GGUF
APEX-I MiniPlus V2.1	14.75 GB / 13.74 GiB	Clover-35B-A3B-Preview-APEX-I-MiniPlus-V2.1-GGUF

Files

File	Description
llama-clover-4838ffb-linux-x86_64-cpu.tar.gz	Linux CPU build, GGML_NATIVE=OFF
llama-clover-4838ffb-linux-x86_64-nvidia-cuda12.tar.gz	CUDA 12 build, archs 80;86;89;90
llama-clover-4838ffb-converter-python.tar.gz	HF-to-GGUF converter, conversion code and gguf-py
llama-clover-4838ffb-windows-x86_64-nvidia-amd-universal-cuda12.8-vulkan-cpu.zip	Windows x86_64 universal: Nvidia CUDA 12.8 sm_50-120, AMD/Intel Vulkan, any CPU
BUILD_INFO.json	Compiler, flags and SHA256 per archive
SHA256SUMS	Checksums
config.json	Machine-readable build descriptor used by Hub download tracking

Coverage

Platform	Target	Backend	Status
Linux x86_64	Any x86-64 CPU	CPU, GGML_NATIVE=OFF	shipped
Linux x86_64	NVIDIA sm_80 / 86 / 89 / 90 (A100, A10G, L40S, RTX 4090, H100/H200)	CUDA 12.0	shipped
Linux x86_64	NVIDIA Blackwell sm_100/120, AMD, Intel	none	not in this build
Windows x86_64	NVIDIA sm_50 through sm_120 (Maxwell to Blackwell)	CUDA 12.8	shipped
Windows x86_64	AMD and Intel GPUs	Vulkan 1.4.363	shipped
Windows x86_64	Any x86-64 CPU	per-CPU DLLs, AVX-512 off	shipped
macOS, ARM64, ROCm on any OS	-	none	not published

Windows is native. The ZIP carries its CUDA 12.8 libraries and uses the Vulkan loader from the GPU driver; extract it and run set PATH=%~dp0bin;%PATH% from the archive root. It ships the same tool set as the Linux archives, including llama-speculative-simple, llama-quantize and llama-imatrix.

The Linux x86_64 archives require GLIBC_2.38 or newer, so Ubuntu 24.04 or equivalent is the safe baseline; older distributions need their own build. Anything outside the table above, including macOS and ARM64, needs a build from the same branch and commit.

Compatibility

• Model: Diffuspace/Clover-35B-A3B-Preview (clover_moe). Works with the BF16 and Q8_0 GGUFs and with third-party clover_moe GGUFs such as the IQ4_XS. It does not load the original safetensors.
• Frontends: not LM Studio or Ollama. Clover needs this clover_moe runtime until the PR merges upstream.

Usage

All runtime archives are dynamically linked and ship llama-cli, llama-server, llama-diffusion-cli, llama-quantize, llama-imatrix, and llama-perplexity. Point the loader at the bundled libraries before running. The archive names changed in this build; the extract directories kept their old names.

CPU:

tar -xzf llama-clover-4838ffb-linux-x86_64-cpu.tar.gz
cd llama-clover-b4838ffb-linux-x64-cpu
export LD_LIBRARY_PATH=$PWD/bin:$LD_LIBRARY_PATH
./bin/llama-cli -m clover-bf16.gguf -c 4096 -p "Hello"

CUDA 12:

tar -xzf llama-clover-4838ffb-linux-x86_64-nvidia-cuda12.tar.gz
cd llama-clover-b4838ffb-linux-x64-cuda12
export LD_LIBRARY_PATH=$PWD/bin:$LD_LIBRARY_PATH
./bin/llama-cli -m clover-bf16.gguf -ngl 99 -fa on -c 4096 -p "Hello"
./bin/llama-speculative-simple -m clover-bf16.gguf --spec-type draft-self --spec-draft-n-max 15 -ctk bf16 -ctv bf16 -n 128 -c 2048 -b 256 -ub 256 -ngl 99

Windows x86_64 (universal ZIP, no separate runtime install):

set PATH=%~dp0bin;%PATH%
llama-completion.exe -m Clover-35B-A3B-Preview.APEX-I-NanoPlus-V2.1.gguf -p "hi" -n 1 -c 128 -b 16 -ub 16 -ngl 8 -no-cnv

Run both lines from the extracted archive root. Drop -ngl 8 for CPU or Vulkan-only use; raise it to offload more layers on a large NVIDIA card.

Conversion

tar -xzf llama-clover-4838ffb-converter-python.tar.gz
python llama-clover-converter/llama.cpp/convert_hf_to_gguf.py Diffuspace/Clover-35B-A3B-Preview --outtype bf16 --outfile clover-bf16.gguf

Quantization and importance matrix

./bin/llama-quantize --allow-requantize clover-bf16.gguf clover-q8_0.gguf Q8_0
./bin/llama-imatrix -m clover-q8_0.gguf -f calibration.raw -o clover.imatrix.gguf --output-format gguf --process-output --no-ppl --no-escape -c 2048 -b 2048 -ub 2048 --chunks 611 -ngl 99

The APEX-I V2.1 releases were produced with this build: llama-quantize --imatrix Clover-VAL-1.25M-I-Matrix.gguf --tensor-type-file <recipe> clover-bf16.gguf <output> Q4_0 64. The recipes carry 853 lines, including the 120 block-diffusion tensors that must stay in F32.

Notes

The CUDA packages require a compatible NVIDIA driver and CUDA 12.x runtime; the Windows package bundles its CUDA 12.8 libraries and uses the Vulkan loader from the GPU driver. llama.cpp is MIT licensed; the Clover support is PR #31306 by Maphist0; the Clover weights are Apache-2.0, Copyright DiffuSpace and the Qwen team.