DeepSeek-Coder-V2-Lite-Instruct-Pyrite-2bit-MLX

🇯🇵 日本語版モデルカード (README_ja.md) はこちら | 🦙 GGUF Format (llama.cpp / Ollama / LM Studio)

Model Summary

DeepSeek-Coder-V2-Lite-Instruct-Pyrite-2bit-MLX is a sensitivity-aware ultra-low-bit MLX quantization of deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct (27 layers, 1 dense MLP + 26 MoE layers with 64 routed experts + 2 shared experts, Multi-Head Latent Attention [MLA], 15.7B total parameters, 2.36B active parameters, ~31.4 GB in BF16).

Built with the Project Pyrite sensitivity-guided activation-weighted quantization pipeline, this model compresses the 16B flagship MoE coding model down to 4.88 GiB (5,241,265,152 bytes), allowing it to run smoothly on any 16GB Apple Silicon Mac (M1/M2/M3/M4) with over 10 GiB of remaining Unified Memory headroom for deep 32k+ token KV cache context.

---

Key Highlights

• Exact Zero-Byte-Overhead Quality Improvement: Maintains the exact same tensor byte budget (5,241,265,152 bytes = 4.8813 GiB) as a uniform 2-bit (group_size=64) baseline down to the single byte ($\Delta = 0\text{ bytes}$).
• Perplexity & Accuracy Gain: Reduces multi-domain validation perplexity from 11.1105 down to 10.9707 and boosts top-1 token agreement from 68.82% to 70.00%.
• Zero Custom Kernel Requirement: Uses stock MLX native affine quantization (mlx.nn.QuantizedLinear), ensuring 100% compatibility with Apple Silicon Metal acceleration and unified memory architecture.
• Preserved Router & Attention Integrity: MoE routing gate matrices (gate.weight) and normalization layers are preserved in raw unquantized precision to maintain stable expert routing, while lm_head is quantized at 4-bit (g64) for high-fidelity vocabulary decoding.

---

Evaluation Benchmark

All models were evaluated under identical tokenized multi-domain held-out sets (Code, English, Japanese):

Model / Quantization Configuration	Model Size (Bytes)	RAM (GiB)	Multi-Domain Val PPL	Code PPL	Top-1 Token Agreement
BF16 Teacher Reference	~31,400,000,000 B	~31.4 GiB	7.0371	1.5854	100.0%
Baseline Uniform 2-bit (g64)	5,241,265,152 B	4.8813 GiB	11.1105	1.8262	68.82%
Pyrite 2-bit (pyrite_down13)	5,241,265,152 B	4.8813 GiB	10.9707 ($\downarrow$)	1.9046	70.00% ($\uparrow$)

Byte Delta: $\Delta = 5,241,265,152 - 5,241,265,152 = \mathbf{0\text{ bytes}}$.
High-sensitivity down_proj layers (Layers 14 through 26) were upgraded from 2-bit to 3-bit, funded by widening 52 MoE gate/up projections from $g64 \to g128$.

---

Quickstart with MLX

Installation

pip install mlx-lm

Generation via CLI

mlx_lm.generate \
  --model masahiroid/DeepSeek-Coder-V2-Lite-Instruct-Pyrite-2bit-MLX \
  --prompt "Write a high-performance Python function using MLX to compute cosine similarity." \
  --max-tokens 512

Python API

from mlx_lm import load, generate

model, tokenizer = load("masahiroid/DeepSeek-Coder-V2-Lite-Instruct-Pyrite-2bit-MLX")

prompt = "Write a complete Rust implementation of a lock-free ring buffer."
messages = [{"role": "user", "content": prompt}]
formatted_prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

response = generate(
    model,
    tokenizer,
    prompt=formatted_prompt,
    max_tokens=512,
    verbose=True
)

---

Technical Report & Citation

This model is part of Project Pyrite: Sensitivity-Aware 2-Bit Quantization on Apple Silicon MLX with Zero-Byte Budget Overhead.

If you use this model or refer to the method, please cite our preprint:

@techreport{nakatsugawa2026pyrite,
  author       = {Nakatsugawa, Masahiro},
  title        = {Project Pyrite: Sensitivity-Aware 2-Bit Quantization on Apple Silicon MLX with Zero-Byte Budget Overhead},
  institution  = {Project Pyrite Technical Report},
  year         = {2026},
  month        = {October},
  doi          = {10.5281/zenodo.23786705},
  url          = {https://doi.org/10.5281/zenodo.23786705}
}

• Permanent DOI: 10.5281/zenodo.23786705
• Author: Masahiro NAKATSUGAWA (中津川 昌弘) / ORCID: 0009-0000-3224-0103