DeepSeek-V4-Flash-Vision-Exp — EXL3 MixedK
Sensitivity-allocated mixed-precision EXL3 quantization of deepseek-ai/DeepSeek-V4-Flash-Vision-Exp: a K2 base with the most quantization-sensitive MoE layers upgraded to K3. ~95 GiB — fits one DGX Spark.
Strategy
Full K2 base (2.0 bpw experts) with **K3 (3.0 bpw) on layers 21, 22, 13,
41, 28, 3**, selected by a per-layer K2-vs-K3 proxy-error scan. K is
self-describing per tensor via trellis shape, so the pack needs no metadata
changes — kernels dispatch per layer at load.
| Layer | K2->K3 gain |
|---|---|
| 21 | 0.081 |
| 22 | 0.081 |
| 13 | 0.081 |
| 41 | 0.081 |
| 28 | 0.081 |
| 3 | 0.081 |
The landscape is near-flat (~0.3% spread across all 43 layers), so this top-6 is a noise-level ordering of near-ties, not a set of distinct outliers. Whether mixed-K measurably beats uniform K2 is pending PPL/KLD.
Source, format, serving
Identical to the K2 base: mixed-format source (MXFP4 experts + FP8 attention) dequanted to BF16 before trellis (4-bit -> 2-bit double quant on experts); MTP/DSpark drafter kept at source MXFP4 verbatim. vLLM EXL3 overlay, one Spark, text-only serve.
Limitations
Not boot-tested; mixed-format load (EXL3 + MXFP4 drafter) unverified; quality
gain over K2 not yet measured (flat landscape suggests it may be small);
DeepSeek license (other). See the K2 card for full detail.
Related
- K2 base
- k3-delta + evidence
- ablit MixedK — uncensored variant
- Downloads last month
- 19
Model tree for vcruz305/DSV4-Flash-Vision-EXL3-MixedK
Base model
deepseek-ai/DeepSeek-V4-Flash-Vision-Exp