Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-AWQ-DSpark

DeepSeek-V4-Flash-Vision-Exp (AutoRound INT4 • W4A16)

Official Solstice-AI 4-Bit Release • 1M Native Context • AutoRound Model-Free Quantization • High-Throughput vLLM & SGLang Serving

Original Architecture by DeepSeek-AI • Upstream Bug Fixes by Unsloth • Quantization & Packaging by Solstice-AI

---

Model Summary

Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-AWQ is the production 4-bit (W4A16) release of DeepSeek's flagship multimodal MoE foundation model, built from Unsloth's verified bug-fixed checkpoint (unsloth/DeepSeek-V4-Flash-Vision-Exp).

Quantization Details:

• Quantizer: Intel AutoRound 0.15.0 in official model-free mode.
• Precision Scheme: W4A16 (4-bit integer weights, 16-bit activations, symmetric, group_size: 128, packing_format: auto_round:auto_gptq).
• Selective Mixed-Precision Layer Preservation:
  • Quantized (33,024 layers): All 256 routed MoE experts across all 43 layers (layers.[0-42].ffn.experts.[0-255].w1, w2, w3) quantized to 4-bit.
  • Preserved Lossless in BF16 (3,021 layers): Attention MLA projections (attn.wq_a, wq_b, wo_a, wo_b, wkv), router gates (ffn.gate), shared experts (ffn.shared_experts.w1, w2, w3), ViT vision tower (vision.blocks.*), multimodal projector (aligner.w1, w2), MTP heads (mtp.*), and embedding / lm_head.
• Native Context Window: Strictly locked to 1,048,576 tokens (1M) with dual-stage YaRN RoPE (factor: 16, original_max_position_embeddings: 65536, beta_fast: 32, beta_slow: 1).
• Speculative Decoding: Native DSpark Markov projection heads and MTP layers are intact in BF16. Standalone companion files (speculative/DSpark-drafter-vision-exp.gguf and mmproj-BF16.gguf) are bundled for cross-ecosystem utility.

---

Official DeepSeek-V4-Flash-Vision-Exp Benchmark Scoreboard

Official benchmark results reported by DeepSeek-AI (evaluated with DeepSeek Harness minimal mode, max reasoning effort, temperature = 1.0, top_p = 0.95):

Benchmark	DeepSeek-V4-Flash-Vision-Exp	DeepSeek-V4-Flash-0731	Opus-4.8
Text Agent Capabilities
Terminal Bench 2.1	83.9	82.7	85.0
NL2Repo	57.7	54.2	69.7
Cybergym	75.3	76.7	78.3
DeepSWE	59.3	54.4	58.0
Toolathlon-Verified	75.9	70.3	76.2
DSBench-Hard	63.6	59.6	71.7
AutomationBench (Public)	25.7	25.1	27.2
Multimodal Agent Capabilities
ApexBench (Pass@1)	36.5	26.2†	39.4
Agents' Last Exam	27.3	25.2†	25.7
Chartography	64.3	—	65.0
ZeroBench (Pass@5)	35.0	—	34.0

† Note: For ApexBench and Agents' Last Exam, text-only DeepSeek-V4-Flash-0731 ignores multimodal elements in the input.

---

Serving & Deployment Recipes

1. vLLM Serving

# AutoRound W4A16 is natively supported by vLLM via auto-round / INC backend
vllm serve Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-AWQ   --quantization auto-round   --max-model-len 131072   --tensor-parallel-size 2   --gpu-memory-utilization 0.92   --trust-remote-code   --port 8000

To enable native DSpark speculative acceleration in vLLM:

vllm serve Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-AWQ   --quantization auto-round   --max-model-len 131072   --tensor-parallel-size 2   --gpu-memory-utilization 0.92   --trust-remote-code   --speculative-config '{"method":"dspark","model":"Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-AWQ","num_speculative_tokens":3,"draft_sample_method":"probabilistic","enable_adaptive_verification":true}'

2. SGLang Serving

python3 -m sglang.launch_server   --model-path Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-AWQ   --quantization auto-round   --context-length 131072   --tp 2   --trust-remote-code   --host 0.0.0.0   --port 30000

To enable native DSpark speculative acceleration in SGLang:

python3 -m sglang.launch_server   --model-path Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-AWQ   --quantization auto-round   --context-length 131072   --tp 2   --trust-remote-code   --speculative-algorithm DSPARK   --mem-fraction-static 0.85

---

Upstream Architecture & Modules

This repository bundles all necessary upstream modules for zero-friction inference:

• inference/ — Official PyTorch reference inference scripts and image processors.
• encoding/ — Native OpenAI-style message formatting and multimodal prompt encoders.
• chat_template.jinja — Official Jinja template for vision + thinking prompt formatting.
• mmproj-BF16.gguf & speculative/DSpark-drafter-vision-exp.gguf — Cross-ecosystem companion files.

Credits & License

• Original Model: DeepSeek-AI
• Base Checkpoint: Unsloth AI
• Quantization & Packaging: Solstice-AI
• License: MIT License