ds4-winhip — ds4 / ds4-server, Windows + HIP 7.2 build (for DeepSeek-V4.1-Flash lecore VQ GGUFs)

A working Windows/HIP port of Baekpica/ds4-dfm-rs (MIT, "the ds4.c authors"), plus the launcher, supervisor and notes used to serve DeepSeek-V4.1-Flash-1017 lecore VQ builds on a Radeon 8060S (gfx1151, HIP SDK 7.2, 128 GB unified memory).

llama.cpp, vLLM and Ollama cannot load these VQ blobs — the routed experts are vector-quantized codebook blobs (vq8x4096/1024) and only the ds4 runtime knows the format. That is what this repo is for: source, build scripts and the exact run command, so you do not have to rediscover them.

Upstream's own README (DwarfStar / ds4 feature tour) is kept at docs/README-upstream-ds4.md.

What is inside

path	what it is
ds4.c, ds4.h, ds4_*.c / .inc / .cuh, crates/, cuda/	the runtime, forked from Baekpica/ds4-dfm-rs at 331b8a7b
hip/win/build_gpu.cmd, build_host.cmd, build_repack.cmd, link_server.ps1	the Windows/HIP build: device code, host code, final link
hip/win/win_compat.c, win_stubs.c, win_stubs_cxx.cpp, ds4_win_compat.h, include/	POSIX/vDSO shims the Windows link needs
hip/win/run_ds41.ps1	the launcher: vision sidecar, Engram dir, MTP/DSpark, KV disk store, sampling defaults
serve/serve_ds41.ps1	unattended supervisor: adopts or relaunches ds4-server, stall watchdog, commit-limit backoff
serve/patch_ds4.py, serve/ds4_boot.sh	the FP4 solar_e2m1_decode fix (CUDA/HIP path) and the Linux boot script used on the H100/H200 boxes
ds4-server.exe, ds4-server/	the built binary that produced the numbers below (Windows x64, HIP 7.2) — usable as-is
docs/, HANDOFF-requant.md	format notes: VQ layout, per-layer byte budgets, kernel constraints (LDS codebook size), re-quant recipe
hf-card-serving.md	the serving section as it appears on the model cards, with the measured gotchas
bench/	raw prefill/decode bench logs (H100/H200 and 8060S)

Excluded to keep the repo small: .git, tests/, gguf-tools/, object files, stale *.old.exe. Upstream history lives at the GitHub link above; this is a source drop of the working tree.

Build (Windows, HIP SDK 7.2)

git clone https://huggingface.co/staccs/ds4-winhip
cd ds4-winhip
.\hip\win\build_gpu.cmd            # device code
.\hip\win\build_host.cmd           # host code
cargo build -p ds4-server          # compiles, then FAILS AT LINK (no error[ lines) — expected
.\hip\win\link_server.ps1          # this produces ds4-server.exe

The link failure after cargo build is normal on this tree: the Rust crate only drives the C build, link_server.ps1 does the final link. A running server keeps its own copy of the exe, so you can rebuild while it serves.

Run

$env:PATH = "C:\Program Files\AMD\ROCm\7.2\bin;$env:PATH"
.\hip\win\run_ds41.ps1 -Gguf D:\models\ds41\DeepSeek-V4.1-Flash-1017-abliterated-pqhaz-lecore-vq11cal.gguf `
                       -Engram D:\models\ds41\deepseek-engram
# OpenAI-compatible API on http://127.0.0.1:8008/v1

The model file is the 106.8 GB lecore VQ GGUF; -Engram points at a directory holding model-00047-of-00048.safetensors and model-00048-of-00048.safetensors from deepseek-ai/DeepSeek-V4.1-Flash-1017 (~203 GB, mapped from disk, not copied). The vision sidecar (0.9 GiB) sits next to the GGUF — see staccs/ds41-sidecars.

Defaults the launcher sets (all overridable by env var):

variable	default	why
DS4_CUDA_COPY_MODEL	1	copies weights into GPU memory (~92 GiB of Windows commit; needs a commit limit above ~100 GB). -NoCopy host-maps instead: no commit, slower per token
DS4_CUDA_WEIGHT_CACHE_LIMIT_GB	100	if one big allocation fails, the per-range cache must hold the whole model
DS4_SERVER_DEFAULT_TEMP / TOP_P / MIN_P	0.6 / 0.95 / 0.05	the card recipe for this quant; T=1.0 untruncated turns into word salad at 14k+ prompts, greedy loops deterministically on long tool turns
DS4_SERVER_DEFAULT_MAX_TOKENS	4096	agent frameworks send no cap; without one a repetition loop runs to the context end (seen: 19.5k tokens, 22 min)
DS4_SERVER_DEFAULT_FREQ_PENALTY	0.05	curbs this quant's runaway answer loops; any penalty disables the DSpark drafter (~8.7 vs ~21 tok/s)
DS4_SERVER_THINK_BUDGET_FILE	...\ds41_think_budget.txt	think-block token budget, read per request; 0 = unlimited
DS4_KV_DISK_DIR / DS4_KV_DISK_SPACE	<modeldir>\kvdisk / 8G	disk KV store: a 14k-token checkpoint is ~30 MB, restored in ~2 s instead of a ~190 s re-prefill

Measured (this tree, Radeon 8060S, gfx1151, 128 GB unified)

metric	value
load to first answer	~6 min (weight copy + ~110 s first-request kernel/graph warm-up)
prefill	135–190 tok/s (real agent prompts vs synthetic)
decode, one stream	14–17 tok/s (21 tok/s with the DSpark drafter when no repetition penalty is set)
decode, 1× H200 NVL	46–54 tok/s (vq10sh12 build)
concurrency	one request at a time; a second queues

Gotchas worth knowing before you debug

• One live KV checkpoint. The server keeps the state after the last prompt and after the last reply. Any other request in between overwrites both and the next turn re-prefills everything (measured: 19.3k tokens, 138 s). While an agent works, nothing else should touch the port.
• Stopping. Kill ds4-server, then wait for Windows commit to drop before relaunching, or the loader thrashes at the commit limit. serve_ds41.ps1 does this for you.
• PowerShell 5.1 reads BOM-less UTF-8 .ps1 as ANSI and mangles the fullwidth bar in <｜...｜> tokens — save scripts that contain them with a BOM.
• usage.prompt_tokens tells you whether a special token in your text was parsed as one id (5 with the chat template) or split into text (18).
• DSpark is disabled by any repetition penalty; accept rate on this quant is 1.0–3.3 of 5 per verification round depending on prompt mix.

License

MIT — the upstream ds4.c tree is MIT ("Copyright (c) 2026 The ds4.c authors"), see LICENSE.