Instructions to use Ttimms/Spark-X2.5-4B-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Ttimms/Spark-X2.5-4B-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Ttimms/Spark-X2.5-4B-NVFP4", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Ttimms/Spark-X2.5-4B-NVFP4", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Ttimms/Spark-X2.5-4B-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Ttimms/Spark-X2.5-4B-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ttimms/Spark-X2.5-4B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Ttimms/Spark-X2.5-4B-NVFP4

SGLang

How to use Ttimms/Spark-X2.5-4B-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Ttimms/Spark-X2.5-4B-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ttimms/Spark-X2.5-4B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Ttimms/Spark-X2.5-4B-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ttimms/Spark-X2.5-4B-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Ttimms/Spark-X2.5-4B-NVFP4 with Docker Model Runner:
```
docker model run hf.co/Ttimms/Spark-X2.5-4B-NVFP4
```

Spark-X2.5-4B — NVFP4 W4A16 (GPTQ)

NVFP4 weight-only (W4A16) quantization of XHToken/Spark-X2.5-4B — 4-bit float weights in 16-element blocks with FP8 block scales, activations in bf16. lm_head and every self_attn.g_proj (the head-wise attention-output gate) kept in bf16, matching XHToken's own INT8 exclusion set. Rounding is GPTQ (Hessian-aware), not RTN — this matters a lot here (see the eval). Produced with llm-compressor 0.13.

2.7 GiB on disk (bf16 base is 7.7 GiB — ~65 % smaller)
The first NVFP4 / 4-bit compressed-tensors build of Spark-X2.5-4B
Serves on vLLM on Blackwell via the out-of-tree Spark2_5 plugin (below); on SM120 the weight-only path decodes through the Marlin kernel to a bf16 GEMM (no native FP4 compute), so the benefit here is footprint, not raw speed
Leaves ~10 GiB of KV headroom on a 16 GB card — room for a large slice of the 1 M-token context window

Evaluation

lm-evaluation-harness, vLLM 0.26 backend (+ Spark2_5 plugin), greedy decoding, median of 3 draws with the range (the harness is non-deterministic run-to-run even at greedy — a single draw is not a reproducible score). HumanEval-instruct pass@1 / create_test (n = 164); MBPP base 3-shot (n = 500). RTX 5070 Ti (Blackwell, SM120).

build	HumanEval-inst	MBPP (3-shot)	size	Δ HE / MBPP vs bf16
bf16 base	79.88 % (79.88–81.10)	48.80 % (48.60–48.80)	7.7 GiB	—
NVFP4-W4A16 GPTQ (this)	75.61 % (73.78–76.22)	46.80 % (46.40–46.80)	2.7 GiB	−4.3 / −2.0 pp
NVFP4-W4A16 RTN	71.34 % (70.73–71.95)	33.20 % (32.60–33.80)	2.7 GiB	−8.5 / −15.6 pp

GPTQ rounding is doing real work on this model — plain RTN loses 8.5 pp on HumanEval and collapses on MBPP (−15.6 pp, 33 %). GPTQ recovers +4.3 pp / +13.6 pp, landing at −4.3 pp HumanEval and −2.0 pp MBPP (the MBPP Wilson intervals overlap the bf16 baseline). For a 4-bit build at 65 % smaller, that is a usable trade. If you want maximum quality retention, XHToken's own FP8 build is the safer choice.

Serving (vLLM + out-of-tree plugin)

Spark2_5ForCausalLM is not in the vLLM model registry. Install the general plugin into the same environment as vLLM:

git clone https://github.com/XHToken/Spark-plugin
pip install -e ./Spark-plugin --no-deps
pip install -U "openai>=2.25.0"

Then serve:

vllm serve Ttimms/Spark-X2.5-4B-NVFP4 \
  --trust-remote-code \
  --enforce-eager \
  --max-model-len 32768 \
  --kv-cache-dtype fp8 \
  --enable-auto-tool-choice --tool-call-parser spark25

On WSL, also set VLLM_USE_V2_MODEL_RUNNER=0 (the V2 runner needs UVA, which WSL disables). --enforce-eager is the safe default on SM120; weight-only NVFP4-W4A16 also runs with CUDA graphs.

Method & provenance

Quantizer: llm-compressor 0.13, GPTQModifier(scheme="NVFP4A16", targets="Linear", dampening_frac=0.1), ignore lm_head + re:.*self_attn\.g_proj + tied embeddings. Calibration: 512 samples of theblackcat102/evol-codealpaca-v1 (code, disjoint from the eval sets), 2048 tok.
Base: XHToken/Spark-X2.5-4B (Spark2_5ForCausalLM, ~4.1 B, hybrid sliding/full attention, head-wise attention-output gate, 1 M context, Apache-2.0).
Built and evaluated on an RTX 5070 Ti (Blackwell, SM120), vLLM 0.26 + the Spark2_5 plugin.

License

Apache-2.0, inherited from XHToken/Spark-X2.5-4B.

Downloads last month: -

Safetensors

Model size

4B params

Tensor type

BF16

Model tree for Ttimms/Spark-X2.5-4B-NVFP4

Base model

XHToken/Spark-X2.5-4B-Base

Finetuned

XHToken/Spark-X2.5-4B

Quantized

(25)

this model