Instructions to use nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP
# Run inference directly in the terminal:
llama cli -hf nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP
# Run inference directly in the terminal:
llama cli -hf nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP
# Run inference directly in the terminal:
./llama-cli -hf nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP
# Run inference directly in the terminal:
./build/bin/llama-cli -hf nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Use Docker

docker model run hf.co/nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

LM Studio
Jan

vLLM

How to use nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Ollama
How to use nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF with Ollama:
```
ollama run hf.co/nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP
```
Unsloth Desktop

How to use nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF with Docker Model Runner:
```
docker model run hf.co/nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP
```

Lemonade

How to use nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Run and chat with the model

lemonade run user.Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF-Q4_0_ROCMFP

List all available models

lemonade list

Hermes Agent

How to use nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF:Q4_0_ROCMFP" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Spark-X2.5-4B — Q4_0_ROCMFP4_STRIX_LEAN (GGUF)

Quantization of XHToken/Spark-X2.5-4B to the ROCmFPX GGUF family, tuned for AMD Strix Halo (Ryzen AI Max / gfx1151).

This GGUF runs ONLY on the ROCmFPX llama.cpp fork — with spark2_5 architecture support (fork main @ c49ebdb + the port patch, see below). Stock llama.cpp, Ollama, and LM Studio cannot load it. The HF/GGUF automated parser does not know the ROCmFPX quant enums and may mislabel this file as "F16" — the real format is Q4_0_ROCMFP4_STRIX_LEAN (4.39 BPW).

Files

File	Size	Notes
`Spark-X2.5-4B-Q4_0_ROCMFP4_STRIX_LEAN.gguf`	2.26 GB (2,260,702,112 bytes)	single file, chat template embedded
`Spark-X2.5-4B-Q4_0_ROCMFP4_STRIX_LEAN.imatrix`	3.57 MB	importance matrix used for this quant (see below)

SHA256 (Spark-X2.5-4B-Q4_0_ROCMFP4_STRIX_LEAN.gguf): f3c3a526d5ef7b8e249ca1b1a092878d047bfefc9e3cb432d21643452cf94699

Quantization details

Source: official BF16 GGUF XHToken/Spark-X2.5-4B-GGUF (Spark-X2.5-4B.gguf, 8.23 GB, verified true BF16)
Preset: Q4_0_ROCMFP4_STRIX_LEAN — 2150.83 MiB, 4.39 bits per weight (smallest-footprint Strix Halo recipe)
Tensor census (290/290 tensors, 1:1 with source):
- Q4_0_ROCMFP4 ×36 — fused attn_qkv of every block (dual per-16 scale, Strix attn-K/V quality)
- Q4_0_ROCMFP4_FAST ×180 — attn_gate, attn_output, ffn_gate/up/down
- F32 ×73 — all norms
- Q5_K ×1 — token_embd (tied embeddings)
Importance matrix: self-generated (the publisher ships none). llama-imatrix over ~29K tokens (57 chunks × 512) sampled proportionally from the eaddario/imatrix-calibration corpus (MIT) — tools_medium (tool-use/agentic prompts), code_medium (code instructions from Magicoder-Evol-Instruct-110K / OpenCoder / McEval) and combined_th_small (Thai) — computed on an F16 copy of the official BF16 GGUF. The .imatrix file is included for reproducibility.

Base model


Model	Spark-X2.5-4B (dense 4.112B)
Architecture	`spark2_5` — hybrid attention 3×SWA(window 512) : 1×full, GQA 16/4 heads, head_dim 256
Context	1,048,576 tokens (`max_position_embeddings`)
Vocabulary	131,072 (tied embeddings), BPE (`tokenizer.ggml.pre = spark2_5`)
Behavior	thinking model — emits `<think>…</think>` reasoning before the answer
License	Apache-2.0 (inherited from base)

Usage (Strix Halo, gfx1151)

Build the ROCmFPX server image from the fork with the spark2_5 patch, then:

docker run -d --name rocmfpx-serve --restart unless-stopped \
  --device /dev/kfd --device /dev/dri \
  --group-add "$(getent group render | cut -d: -f3)" \
  --group-add "$(getent group video  | cut -d: -f3)" \
  -p 8080:8080 -v /models:/models:ro \
  -e HSA_OVERRIDE_GFX_VERSION=11.5.1 \
  -e GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \
  rocmfpx:server \
  -m /models/Spark-X2.5-4B-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  -ngl 999 -fa on -c 32768 -b 512 -ub 512 \
  -ctk q8_0 -ctv q8_0 \
  --jinja

GET :8080/health when ready. The model has no MTP/draft head — do not pass --spec-* flags. Weights are ~2.3 GB and SWA keeps the KV cache small, so this fits Strix Halo's unified memory comfortably. Expect fast decode via the HIP/Vulkan FP4 paths (CPU-only hosts will be slow — the format targets the GPU).

Verification

The artifact was verified end-to-end on the quantize box: loads and generates coherently through the ported spark2_5 graph on both the x86_64 ROCmFPX binaries (QEMU) and an aarch64 CPU build — e.g. correct <think>-style reasoning and answers ("The capital of France is Paris.").
Confirmed working on real hardware: runs on an AMD Strix Point iGPU via the ROCmFPX fork. Strix Halo (gfx1151) throughput numbers are not yet measured.

Acknowledgements

Base model: XHToken/Spark-X2.5-4B (Apache-2.0)
Format & runtime: ROCmFPX llama.cpp fork
spark2_5 support in the fork comes from an upstream llama.cpp port (PR #27868, b10828) — patch against fork c49ebdb available on request

Downloads last month: 65

GGUF

Model size

4B params

Architecture

spark2_5

Hardware compatibility

4-bit

Model tree for nanash66/Spark-X2.5-4B-ROCmFP4-STRIX_LEAN-GGUF

Base model

XHToken/Spark-X2.5-4B-Base

Finetuned

XHToken/Spark-X2.5-4B

Quantized

(41)

this model