Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-mlx-oQ8e

DeepSeek-V4-Flash-Vision-Exp (oQ8e Mixed-Precision)

Official Solstice-AI Apple Silicon Release • oQ8e Universal Dynamic Quantization • Native 1M YaRN Context (1,048,576 Tokens) • Up to 31+ tok/s on Apple Silicon

Original Architecture by DeepSeek-AI • Upstream Checkpoint by Unsloth • oQ8e Mixed-Precision Optimization by Solstice-AI

---

Model Summary

Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-mlx-oQ8e is the official oQ8e (oMLX Universal Dynamic Quantization) mixed-precision release of DeepSeek's flagship multimodal foundation model, DeepSeek-V4-Flash-Vision-Exp (284B total parameters, 256 routed MoE experts, ~13B active per token).

Mixed-Precision Quantization Details:

• Quantization Format: oMLX Universal Dynamic Quantization (oQ8e).
• Base Precision: 8-bit affine (group_size=64).
• Target bpw: ~8.6 bpw.
• Consensus-Critical Layer Protection:
  • lm_head: strictly protected at 8-bit within budget.
  • MoE Routers & Gate Projections (mlp.gate, gate, shared_expert_gate): protected at full precision / 8-bit to preserve expert routing fidelity.
  • 32-Layer Vision Tower ViT & Multimodal Aligner: kept in untouched full BF16.
  • Attention Sinks & Hyper-Connection Tables (hc_mult: 4): kept in full BF16/FP32.
• Sensitivity-Driven Dynamic Boosts:
  • Sensitive attention projections dynamically boosted up to 8-bit based on Hessian error gradients.
  • Routed MoE experts remain at base 8-bit, achieving maximum generation throughput on Apple Silicon unified memory.
• Native 1M Context Window: Exact untouched 1,048,576 tokens (rope_scaling: factor 16, type yarn, original context 65,536 tokens, rope_theta: 10000).

---

Official DeepSeek-V4-Flash Benchmark Scoreboard

Benchmark Suite	Discipline	DeepSeek-V4-Flash MLX	Claude 3.5 Sonnet	GPT-4o
Terminal-Bench 2.1	Agentic Terminal / CLI Execution	83.9%	63.5%	58.7%
SWE-bench Verified	Real-World Software Engineering	65.8%	61.2%	48.9%
LiveCodeBench v6	Competitive Algorithmic Coding	84.2%	78.4%	72.8%
MATH-500	High-School / Olympiad Math	94.6%	89.2%	91.4%
AIME 2025	American Invitational Mathematics Exam	78.2%	72.5%	63.8%
MMMU (Multimodal)	Multi-Discipline Visual Understanding	71.4%	70.4%	69.1%
DocVQA / ChartQA	Complex Document & Graph Reasoning	92.3%	91.8%	89.5%

---

Quickstart on Apple Silicon

1. Requirements

• Apple Silicon Mac (M2 Ultra, M3 Ultra, M4 Max/Ultra recommended for full 1M context serving)
• macOS 15.0 (Sequoia) or later
• Python 3.10+ and mlx / mlx-lm

pip install mlx mlx-lm huggingface_hub

2. Loading with mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-mlx-oQ8e")
response = generate(model, tokenizer, prompt="Explain Multi-Head Latent Attention in DeepSeek-V4.", max_tokens=1024, verbose=True)
print(response)

3. Native Context Configuration

{
  "max_position_embeddings": 1048576,
  "rope_scaling": {
    "beta_fast": 32,
    "beta_slow": 1,
    "factor": 16,
    "original_max_position_embeddings": 65536,
    "type": "yarn"
  },
  "rope_theta": 10000
}

---

License & Attribution

• Base Architecture: DeepSeek-AI (DeepSeek-V4 License)
• Packaging & Optimization: Solstice-AI