fraserprice/DeepSeek-V4-Flash-Abliterated

DeepSeek V4 Flash Abliterated

Ported to vLLM for massive throughput on RTX Pro 6000 — by Fraser Price

This is probs the best uncensored model you can run on 2 x RTX Pro 6000 at high throughput (source: ✨ my opinion ✨). Served on the voipmonitor vLLM fork it runs ~13× prefill and ~2.8× single-stream decode faster than the widely available abliterated GGUF versions.

---

Quick start (one command)

Requires Docker with the NVIDIA Container Toolkit, and ≥2 GPUs (see Hardware).

bash run.sh

run.sh defaults to this repo, so no arguments are needed. Everything is configured via env vars (see the table below) — e.g. GPUS=0,1,2,3 TP=4 bash run.sh to run on 4 GPUs.

run.sh (included) will:

1. Download the weights (~156 GB) into your Hugging Face cache (\~/.cache/huggingface, or $HF_HOME/$HF_HUB_CACHE) if not already present.
2. Pull the inference image if not already present.
3. Serve the model on http://localhost:8000/v1 (OpenAI API spec).

Optional — MTP speculative decoding (~1.8× single-stream decode speed). The checkpoint ships its MTP layer; enable it with MTP=2 bash run.sh (see the table below).

Override anything via env vars (all optional):

Variable	Default	Meaning
GPUS	0,1	GPU indices to use (comma-separated)
TP	2	Tensor-parallel size — set to the number of GPUs in GPUS
MTP	(off)	MTP speculative-decoding draft tokens (e.g. 2); empty disables it
PORT	8000	API port
MAX_MODEL_LEN	262144	Context length
GPU_MEM_UTIL	0.92	Fraction of VRAM to use
HF_REPO	fraserprice/DeepSeek-V4-Flash-Abliterated	Hugging Face repo to download
MODEL_DIR	(HF cache)	Serve from a specific local dir instead of downloading
SERVED_NAME	DeepSeek-V4-Flash-Abliterated	Model name exposed by the API
IMAGE	(pinned voipmonitor/vllm build)	Inference container image

Example on 4 GPUs:

GPUS=0,1,2,3 TP=4 bash run.sh

---

Hardware/Image

Tuned for NVIDIA RTX Pro 6000 Blackwell (96 GB, sm_120). It will run on other Blackwell-class GPUs, but the numbers below and the default flags assume RTX Pro 6000. It needs roughly 170 GB of total VRAM for weights + KV cache at long context, split across GPUs by tensor parallelism.

Setup	Works?
2 × RTX Pro 6000 Blackwell (96 GB) — TP=2	✅ reference config
4 × RTX Pro 6000 Blackwell — TP=4	✅ (more KV headroom / throughput)
4 × 80 GB (e.g. H100/H200) — TP=4	✅ (untested here, I believe can just use base vLLM)
1 × DXG Spark	❓ idk why you didn't get an RTX Pro lmao (but lmk if you manage to run this and I can add to docs)
1 × anything	❌ (blessed are the poor in VRAM 🙏)

The serving image targets Blackwell (sm_120) and CUDA 13.2. The DeepSeek-V4 architecture (sparse attention / lightning indexer, MLA, MTP, etc.) requires this image to run on RTX Pro. See local-inference-lab/rtx6kpro for image, bench, and serving details.

---

Performance

Measured under load on RTX Pro 6000 Blackwell (connected via PCIe 5.0), 128 generated tokens/request, TP = number of GPUs. MTP is speculative decoding (— = off, 2 = 2 draft tokens). PP / TG are prefill / per-request decode throughput (tok/s); Total is aggregate end-to-end throughput across all concurrent requests (tok/s). Rows are paired off/on so the MTP decode gain is directly comparable.

GPUs	MTP	Prompt	Conc	TTFT	PP	TG	Total
2	—	1,000	1	132 ms	7,613	108.6	870
2	2	1,000	1	143 ms	7,005	176.2	1,310
2	—	1,000	3	275 ms	4,190	76.7	1,746
2	2	1,000	3	332 ms	3,746	125.7	2,435
2	—	10,000	1	1,036 ms	9,658	108.0	4,581
2	2	10,000	1	1,069 ms	9,357	181.6	5,729
2	—	10,000	3	2,138 ms	5,302	57.1	6,600
2	2	10,000	3	2,205 ms	5,146	85.3	7,496
2	—	100,000	1	11,575 ms	8,640	107.6	7,850
2	2	100,000	1	11,933 ms	8,380	196.2	7,959
2	—	100,000	3	23,579 ms	5,153	33.0	8,161
2	2	100,000	3	24,349 ms	4,991	59.6	8,065
4	—	1,000	1	119 ms	8,437	117.9	946
4	2	1,000	1	128 ms	7,858	216.5	1,584
4	—	1,000	3	255 ms	4,539	98.0	2,175
4	2	1,000	3	507 ms	3,276	129.7	1,996
4	—	10,000	1	842 ms	11,886	118.4	5,293
4	2	10,000	1	873 ms	11,459	223.9	7,034
4	—	10,000	3	1,741 ms	6,521	69.0	8,041
4	2	10,000	3	1,813 ms	6,257	108.7	9,216
4	—	100,000	1	9,530 ms	10,494	118.5	9,445
4	2	100,000	1	9,856 ms	10,146	227.4	9,614
4	—	100,000	3	19,226 ms	6,299	40.0	10,028
4	2	100,000	3	20,037 ms	6,049	73.1	9,824

MTP=2 lifts single-stream decode by ~1.6–1.9× (e.g. 108 → 196 tok/s at 100k on TP=2) for a negligible TTFT cost.

---

How it was made

DeepSeek-V4-Flash's published abliterations existed only as a GGUF for the bespoke ds4/llama.cpp engines, which leave Blackwell hardware massively underutilised.

Rather than re-run abliteration from scratch, the refusal subspace was simply recovered and re-applied from an existing abliteration:

1. Recover the refusal directions. The rank-3 refusal subspace in the 4096-dim residual stream was extracted from the delta between huihui-ai's abliterated GGUF and the clean base.
2. Re-apply to the official weights. The same rank-3 projection was applied to every residual-writing matrix of the official checkpoint — attention output (attn.wo_b) and shared-expert down-projection (shared_experts.w2) across all 43 layers and the MTP layer (88 tensors total). Each tensor is dequantised → projected → requantised, with a fixed-point solver that compensates for FP8 requantisation so the intended projection survives quantisation.
3. Leave everything else untouched. The FP4-quantised routed experts and all gate/up projections are byte-identical to the official release; only the 88 FP8 residual-writing matrices change.

Validation: on every edited tensor the achieved projection matched target within ~0.002, and all 33,880 non-target tensors were verified byte-identical to the base. The served model drops refusals while keeping correct tool-calling and coherent step-by-step reasoning. No formal evals yet — tested manually; it should match the base huihui-ai abliteration at far higher speed.

---

Credits

• deepseek-ai/DeepSeek-V4-Flash — the base model (MIT).
• huihui-ai — the original abliteration (Huihui-DeepSeek-V4-Flash-abliterated-ds4-GGUF), from which the refusal directions were derived (MIT).
• voipmonitor/vllm — the Blackwell/CUDA-13.2 vLLM build that runs the DeepSeek-V4 architecture on RTX Pro.
• local-inference-lab/rtx6kpro — the serving recipe (backends, flags, and benchmarks) this config is based on.

This checkpoint contains no huihui-ai weights — only the official DeepSeek-V4-Flash weights with a refusal-direction projection applied.

Disclaimer

This is an uncensored model: the refusal subspace has been removed, so it will not decline requests the way the base model does. It is released as-is, for research/lawful use only.

• Not a safety-aligned model. Because guardrails have been removed, outputs may be inaccurate, offensive, or harmful. Apply your own filtering, review, and safeguards before relying on or deploying it.
• No warranty. The weights and scripts are provided "as is", without warranty of any kind, express or implied. You use them entirely at your own risk.
• You are responsible for what you generate. The author does not endorse, encourage, or accept any responsibility or liability for how this model is used, including any unlawful, harmful, or unethical use. Outputs do not reflect the views of the author.

License

MIT, inheriting from the base model and the abliteration source.

---

Built by Fraser Price — @fraserpricee