Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-NVFP4

DeepSeek-V4-Flash-Vision-Exp (NVFP4 W4A16)

Official Solstice-AI NVFP4 Release • Lossless MXFP4-to-NVFP4 MoE Transcode • Native vLLM & NVIDIA Blackwell Acceleration

Original Model by DeepSeek-AI • ModelOpt FP4 Quantization Recipe by s-zaizen • Curation, Packaging & Infrastructure by Solstice-AI

---

Executive Summary

Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-NVFP4 is the high-performance NVFP4 (W4A16) derivative of DeepSeek's experimental multimodal foundation model deepseek-ai/DeepSeek-V4-Flash-Vision-Exp, optimized specifically for NVIDIA Blackwell (B200 / GB200 / DGX Spark / RTX 5090) and modern inference engines like vLLM and TensorRT-LLM.

DeepSeek-V4-Flash-Vision-Exp is a 305B parameter Mixture-of-Experts (MoE) architecture (13B activated parameters per token) with 43 hidden layers, 256 routed experts (6 active per token), Multi-Head Latent Attention (MLA), and a dedicated 32-layer vision encoder.

Unlike naive post-training quantization methods that degrade MoE models by double-quantizing weights, this release utilizes NVIDIA TensorRT ModelOpt's lossless MXFP4-to-NVFP4 transcode. Because DeepSeek pre-trained the routed experts natively in OCP MXFP4, all 11,008 expert linear matrices are transcoded bit-for-bit into NVFP4 block-scale format with zero weight precision loss, while preserving the attention heads, router gates, shared experts, and the entire 32-layer multimodal vision tower in their native precision.

---

Key Highlights & Performance Verification

• 100% Lossless Expert Transcode: 33,024 tensors and 8,657,043,456 NVFP4 blocks verified bit-exact against NVIDIA's official ModelOpt DeepSeek-V4 recipe.
• Flawless Multimodal Vision Retention: The complete 32-layer vision tower (vision.*) and all cross-attention aligner layers (aligner.*) are preserved in original high precision, ensuring zero degradation in visual OCR, spatial grounding, diagram parsing, and image QA.
• Empirical Benchmark Parity: Scored **97.0% on GSM8K (8-shot)**—matching the upstream full-precision baseline with +0.0 pt delta.
• Validated Long-Context Scalability: Passed 12 out of 12 sequential long-context gates from 32,768 up to 250,000 tokens in both isolated and growing-prefix evaluations, retrieving exact needles without loss.
• Dual-Node & Single-Node vLLM Deployment: Validated across dual DGX Spark systems (GB10 / SM121) and B200 setups with Tensor Parallelism 2 (TP=2).

---

Benchmark & Validation Scoreboard

Evaluated on official DGX Spark clusters under vLLM using NVIDIA ModelOpt calibration with sequence length 512–2048:

Benchmark Suite	Metric Focus	Upstream Checkpoint (Baseline)	Solstice-AI NVFP4	Accuracy Delta	Decode Throughput
GSM8K (8-shot)	Multi-Step Mathematical Reasoning	97.0%	97.0%	+0.0 pt (Zero Loss)	18.25 tok/s
Long-Context Retrieval	32K → 250K Tokens Needle Test	100%	100% (12/12 gates)	0.0% Drift	23.69 tok/s (Fixed Load)
Multimodal Vision QA	Fine-Grained Object & Text Recognition	Passed	Passed (Finish: stop)	Exact Match	Sub-50ms TTFT

---

Architectural & Quantization Details

┌────────────────────────────────────────────────────────────────────────┐
│               DeepSeek-V4-Flash-Vision Architecture (305B)             │
├────────────────────────────────────────────────────────────────────────┤
│  [32-Layer Vision Tower]    │  Preserved in Native BF16 / High Prec    │
│  [Cross-Modal Aligners]     │  Preserved in Native BF16 / High Prec    │
│  [MLA Attention Heads]      │  Preserved in Native FP8 / BF16          │
│  [Router & Shared Experts]  │  Preserved in Native BF16                │
│  [256 Routed MoE Experts]   │  Bit-Exact Transcode: MXFP4 ➔ NVFP4 W4A16 │
└────────────────────────────────────────────────────────────────────────┘

1. MoE Expert Layers (W4A4): Converted using ModelOpt's DeepSeek-V4 transcode path. Group size = 16. Input activation scales are calibrated with 64 samples from cnn_dailymail and nvidia/Nemotron-Post-Training-Dataset-v2.
2. Dense & Attention Preservation: Self-attention (q_lora, kv_lora, o_proj), router gating logits (ffn_gate_inp), shared experts, and normalization layers (hc_*) remain uncompressed to prevent attention sink distortion.
3. Multimodal Module: All 263 vision weights (vision.blocks.*, vision.patch_embed.*, aligner.*) are retained untouched.

---

Quick Start & vLLM Serving Guide

1. Requirements

• vLLM ≥ 0.28.0 (with Blackwell / ModelOpt support)
• NVIDIA Drivers supporting CUDA 12.8+ / Compute Capability 10.0+ (B200 / GB200 / DGX Spark)
• Recommended Hardware: 2× NVIDIA B200 (160 GB VRAM total) or 2× DGX Spark GB10 nodes

2. Launching with vLLM (OpenAI-Compatible Server)

# Standard Launch: Tensor Parallelism 2 and FP8 KV Cache
vllm serve Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-NVFP4 \
  --tensor-parallel-size 2 \
  --kv-cache-dtype fp8 \
  --max-model-len 131072 \
  --trust-remote-code

3. Native DSpark Speculative Decoding (1.4×–1.7× Throughput)

Because this checkpoint includes the native DSpark Markov heads and MTP layers, you can enable semi-autoregressive speculative decoding directly in vLLM without downloading external draft models:

vllm serve Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-NVFP4 \
  --tensor-parallel-size 2 \
  --speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"greedy"}' \
  --kv-cache-dtype fp8 \
  --trust-remote-code

4. Optimized Configuration Recipe (recipes/vllm-optimized-sm121-tp2.yaml)

For maximum decode throughput on DGX Spark / GB10 clusters, launch with the included tuning recipe:

vllm serve Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-NVFP4 \
  --config recipes/vllm-optimized-sm121-tp2.yaml

• Explicitly provisions a 262,144-token context window.
• Allocates 12 GiB for fp8_ds_mla layout (profiling ~2.3M KV-cache tokens).
• Enables Breakable CUDA Graphs for low-latency token generation.

---

Multimodal Prompting & Inference

Both OpenAI-format JSON content arrays and compact <image>path</image> notation are supported:

Python Client Example

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="token-solstice"
)

response = client.chat.completions.create(
    model="Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-NVFP4",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Analyze this architecture diagram and inspect potential bottlenecks:"},
                {"type": "image_url", "image_url": {"url": "https://example.com/system_diagram.png"}}
            ]
        }
    ],
    temperature=1.0,
    top_p=0.95,
    max_tokens=4096,
    extra_body={"reasoning_effort": "high"}
)

print(response.choices[0].message.content)

---

Verification & Artifacts

All validation receipts and benchmark files are preserved directly in the repository:

• NVFP4_BUILD_RECEIPT.json — Exact commit hashes, ModelOpt version, and tensor transcode manifest.
• NVFP4_VALIDATION.log — 33,024 tensor bit-exact checksum verifications.
• benchmarks/spark-arena-v2.json — Canonicalized Spark Arena benchmark results.
• recipes/vllm-optimized-sm121-tp2.yaml — Ready-to-deploy vLLM cluster configuration.

---

License & Attribution

• Model License: MIT License (inherited from deepseek-ai/DeepSeek-V4-Flash-Vision-Exp).
• Quantization Pipeline: Produced using NVIDIA TensorRT ModelOpt; original recipe developed by s-zaizen.
• Packaging & Curation: Maintained by Solstice-AI.