Instructions to use YTan2000/Spark-X2.5-4B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use YTan2000/Spark-X2.5-4B-GGUF with Grok:

# No code snippets available yet for this library.

# To use this model, check the repository files and the library's documentation.

# Want to help? PRs adding snippets are welcome at:
# https://github.com/huggingface/huggingface.js

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use YTan2000/Spark-X2.5-4B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf YTan2000/Spark-X2.5-4B-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf YTan2000/Spark-X2.5-4B-GGUF:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf YTan2000/Spark-X2.5-4B-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf YTan2000/Spark-X2.5-4B-GGUF:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf YTan2000/Spark-X2.5-4B-GGUF:BF16
# Run inference directly in the terminal:
./llama-cli -hf YTan2000/Spark-X2.5-4B-GGUF:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf YTan2000/Spark-X2.5-4B-GGUF:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf YTan2000/Spark-X2.5-4B-GGUF:BF16

Use Docker

docker model run hf.co/YTan2000/Spark-X2.5-4B-GGUF:BF16

LM Studio
Jan

vLLM

How to use YTan2000/Spark-X2.5-4B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "YTan2000/Spark-X2.5-4B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "YTan2000/Spark-X2.5-4B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/YTan2000/Spark-X2.5-4B-GGUF:BF16

Ollama
How to use YTan2000/Spark-X2.5-4B-GGUF with Ollama:
```
ollama run hf.co/YTan2000/Spark-X2.5-4B-GGUF:BF16
```
Unsloth Desktop

How to use YTan2000/Spark-X2.5-4B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf YTan2000/Spark-X2.5-4B-GGUF:BF16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "YTan2000/Spark-X2.5-4B-GGUF:BF16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use YTan2000/Spark-X2.5-4B-GGUF with Docker Model Runner:
```
docker model run hf.co/YTan2000/Spark-X2.5-4B-GGUF:BF16
```

Lemonade

How to use YTan2000/Spark-X2.5-4B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull YTan2000/Spark-X2.5-4B-GGUF:BF16

Run and chat with the model

lemonade run user.Spark-X2.5-4B-GGUF-BF16

List all available models

lemonade list

Hermes Agent

How to use YTan2000/Spark-X2.5-4B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf YTan2000/Spark-X2.5-4B-GGUF:BF16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default YTan2000/Spark-X2.5-4B-GGUF:BF16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use YTan2000/Spark-X2.5-4B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf YTan2000/Spark-X2.5-4B-GGUF:BF16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "YTan2000/Spark-X2.5-4B-GGUF:BF16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Spark-X2.5-4B — GGUF quants

GGUF conversions of XHToken/Spark-X2.5-4B (Apache-2.0, xAI Spark lineage — parent xai/grok-code-fast-1). All credit for the model goes to XHToken and xAI; this repo only provides the converted/quantized weights.

Spark-X2.5-4B is a compact reasoning model: 4.11B params, hybrid sliding-window attention (1:3 ratio, 4 window layers per 1 global layer, window 128), native 1M-token context.

Files

Path	Quant	Size	Notes
`iq4_xs/Spark-X2.5-4B-IQ4_XS.gguf`	IQ4_XS (4.25 bpw)	2.22 GiB	best general quality-per-byte
`tq3_4s/Spark-X2.5-4B-TQ3_4S.gguf`	TQ3_4S arch-tuned (4.81 bpw)	2.35 GiB	fastest hard-coding variant (Hard86 46.5%) — requires turbo-tan/llama.cpp-tq3
`bf16/Spark-X2.5-4B-BF16.gguf`	BF16	8.22 GiB	reference quality

Required runtime

IQ4_XS / BF16: any recent upstream llama.cpp with spark2_5 support (PR #27868).
TQ3_4S: turbo-tan/llama.cpp-tq3 fork (custom turbo four-scale quant type), branch main ≥ 46b6224a0.

Quick start

llama-server -m Spark-X2.5-4B-IQ4_XS.gguf --jinja -ngl 99 -c 32768 -fa on

--jinja is mandatory — the Spark2.5 chat template is custom; without the flag llama.cpp refuses to load ("custom template is not supported").

Thinking control

The model is a reasoning model and emits <think> blocks by default. With thinking enabled it can consume the whole token budget on reasoning and return empty content for code-style prompts. Disable per-request:

{"messages": [...], "chat_template_kwargs": {"enable_thinking": false}}

Direct-answer and code-eval workloads should run with thinking off (or a very large max_tokens). All scores below were measured with thinking disabled.

Benchmarks

RTX 3090, temp 0, official scorers (EvalPlus, Hard86 test harness), r=3 for speed.

Speed (llama-bench, ngl=99)

Quant	pp512 (t/s)	tg128 (t/s)
BF16	6724	93.3
IQ4_XS	9102	213.6
TQ3_4S (arch-tuned)	7736	215.2

Quality (thinking off)

Suite	BF16	IQ4_XS	TQ3_4S
Hard86 (86 tests)	54.7%	44.2%	46.5%
HumanEval	78.0%	73.8%	71.3%
HumanEval+	73.8%	70.1%	67.7%
MBPP	72.5%	69.6%	68.8%
MBPP+	62.2%	61.4%	59.3%
Internal quality suite (overall)	75.7	74.1	73.9

How to choose: IQ4_XS for general use (best EvalPlus numbers, smallest 4-bit file). TQ3_4S for hard coding tasks at the same decode speed (+2.3pp Hard86, −2.5pp HumanEval, fork runtime required). BF16 when you need reference quality and can pay 2.3× in speed.

TQ3_4S arch-tuning note

The stock uniform TQ3_4S recipe performs poorly on this architecture (HumanEval 43.3%). The shipped file uses an arch-tuned tensor recipe — fused attn_qkv and ffn_down at Q5_K, attn_output at Q4_K, attn_gate at F16, rest turbo four-scale — developed and validated in this campaign (HumanEval 43.3 → 71.3, Hard86 17.4 → 46.5). attn_output specifically must stay Q4_K: Q5_K there collapses decode speed (215 → 86 t/s) by breaking the fused kernel path.

Conversion recipe

# IQ4_XS
python convert_hf_to_gguf.py <source-dir> --outtype bf16 --outfile Spark-X2.5-4B-BF16.gguf
llama-quantize Spark-X2.5-4B-BF16.gguf Spark-X2.5-4B-IQ4_XS.gguf IQ4_XS

# TQ3_4S (arch-tuned, turbo-tan/llama.cpp-tq3)
llama-quantize --tensor-type "attn_qkv=q5_k" --tensor-type "attn_gate=f16" \
  --tensor-type "ffn_down=q5_k" --tensor-type "attn_output=q4_k" \
  Spark-X2.5-4B-BF16.gguf Spark-X2.5-4B-TQ3_4S.gguf TQ3_4S