ricursiveAI

/

Ripple-65B-Preview-NVFP4

Like 26

Follow

Ricursive Intelligence 209

Thanks for the Spark recipe. I've been serving this checkpoint with the DFlash2 draft on a single GB10 box (128GB unified) and wanted to share a few things that weren't obvious:

• Set temperature explicitly. There are no sampling defaults in generation_config.json, so servers fall back to temperature 1.0. At 1.0 non-English answers (Korean in my case) turned into gibberish. Temperature 0, as in spark.py, is clean.
• HTTP server. spark.py is an offline engine. Wrapping Spark and exposing its OpenAIServingChat through FastAPI on the engine's own event loop gives a working OpenAI-compatible API (streaming, tool calls, reasoning_content). Create the engine under if __name__ == "__main__":, otherwise the spawned scheduler re-imports the script and starts a second engine.
• Draft tokens 16 > 32 on GB10. Same 5 prompts, temperature 0: code 24.5 vs 23.2 tok/s, prose 9.0 vs 7.4 tok/s. 64 was slower on both.
• Radix cache works with DFlash. With --disable-radix-cache removed, an agent-style next turn on a 5.8K-token prompt went from 10.2s to 1.0s to first token. Prefill on GB10 is only ~450–500 tok/s, so this matters a lot for agents.
• 3 concurrent requests (--max-running-requests 3, decode CUDA graphs for bs 1–3): 3 users at once went from 23.9 to 52.3 tok/s total, with no change for a single user.

Result: 34–36 tok/s on code, ~0.4s to first token, and a Hermes agent fixed 4 failing tests in a small repo in 1m 36s, all on the one box.