rlaope/Ripple-65B-Spark-Server

Ripple-65B Spark Server

A tuned, one-command OpenAI-compatible server for Ripple-65B Preview (NVFP4) on a single NVIDIA GB10 / DGX Spark box.

This repo holds no weights. setup.sh pulls the official NVFP4 checkpoint and the DFlash2 draft from Ricursive, plus their Spark inference code, then applies a small patch.

Quickstart

git clone https://huggingface.co/rlaope/Ripple-65B-Spark-Server && cd Ripple-65B-Spark-Server
./setup.sh                 # pinned venv + ~65 GB of weights, into ~/ripple
~/ripple/ripple.sh         # first start ~11 min (kernel compile), later ~2 min

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:30000/v1", api_key="none")
r = client.chat.completions.create(model="ripple", messages=[{"role": "user", "content": "Write an LRU cache in Python. Code only."}])
print(r.choices[0].message.content)

Streaming, tool calls and reasoning_content work as in Ricursive's server. GET /stats returns live server-wide tok/s, GPU and memory. Set RIPPLE_HOST=0.0.0.0 to serve your LAN (there is no auth).

What this changes vs the official Spark recipe

Change	Why	Measured on GB10
HTTP server around spark.py	spark.py is an offline engine with no API	OpenAI-compatible /v1/chat/completions
temperature defaults to 0	the checkpoint ships no sampling defaults; at 1.0 non-English answers fell apart	clean output
draft tokens 32 → 16	most steps accept far fewer than 32	prose 7.4 → 9.0 tok/s, code 23.2 → 24.5
radix (prefix) cache on	agents resend the same long prefix every turn; prefill is only ~500 tok/s	next turn TTFT 10.2 s → 1.0 s (5.8K prompt)
3 concurrent requests	the box is bandwidth-bound, batching is nearly free	3 users: 23.9 → 52.3 tok/s total

Knobs (env): RIPPLE_DRAFT_TOKENS, RIPPLE_MAX_RUNNING, RIPPLE_RADIX, RIPPLE_PORT, RIPPLE_HOST.

Results (single GB10, temperature 0)

Code generation	20–36 tok/s (avg 26.5 over 3 prompts; 34–36 on an LRU-cache prompt)
Prose	~9 tok/s
Time to first token	~0.4 s (short prompts)
Agent run	Hermes Agent fixed 4 failing tests in a small repo in 1m 36s, 14 tool calls, ~44 W GPU power

The catch: Ripple is dense, so every token reads all 65B weights. On this bandwidth-bound box that caps plain decoding near ~5 tok/s; the speed comes from DFlash2 speculative decoding, which helps code far more than prose.

Files

• setup.sh one-time install, ripple.sh start/stop, serve_http.py the server, spark.py.diff the patch to Ricursive's spark.py
• generation_config.json greedy sampling defaults, bench_speed.py the 5-prompt speed benchmark

Model, weights and Spark code: © Ricursive Intelligence, Apache 2.0. This kit: Apache 2.0.