Instructions to use Zrald/zralddeepseekv4.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Zrald/zralddeepseekv4.1 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Zrald/zralddeepseekv4.1:Q8_0
# Run inference directly in the terminal:
llama cli -hf Zrald/zralddeepseekv4.1:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Zrald/zralddeepseekv4.1:Q8_0
# Run inference directly in the terminal:
llama cli -hf Zrald/zralddeepseekv4.1:Q8_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Zrald/zralddeepseekv4.1:Q8_0
# Run inference directly in the terminal:
./llama-cli -hf Zrald/zralddeepseekv4.1:Q8_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Zrald/zralddeepseekv4.1:Q8_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Zrald/zralddeepseekv4.1:Q8_0

Use Docker

docker model run hf.co/Zrald/zralddeepseekv4.1:Q8_0

LM Studio
Jan

vLLM

How to use Zrald/zralddeepseekv4.1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Zrald/zralddeepseekv4.1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Zrald/zralddeepseekv4.1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Zrald/zralddeepseekv4.1:Q8_0

Ollama
How to use Zrald/zralddeepseekv4.1 with Ollama:
```
ollama run hf.co/Zrald/zralddeepseekv4.1:Q8_0
```
Unsloth Desktop
Docker Model Runner
How to use Zrald/zralddeepseekv4.1 with Docker Model Runner:
```
docker model run hf.co/Zrald/zralddeepseekv4.1:Q8_0
```

Lemonade

How to use Zrald/zralddeepseekv4.1 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Zrald/zralddeepseekv4.1:Q8_0

Run and chat with the model

lemonade run user.zralddeepseekv4.1-Q8_0

List all available models

lemonade list

Atomic Chat

Zrald DeepSeek-V4.1-Flash-748B Three-Category Quantized (GGUF Release)

📖 Research White Paper: Read our full empirical study and mathematical proof on AMD Instinct MI300X hardware:
📄 Read Whitepaper (PDF) | 🌐 View Online in Browser | 📥 Direct Download

High-efficiency, hardware-benchmarked GGUF releases of DeepSeek-V4.1-Flash (748B MoE + Engram) evaluated on real AMD Instinct™ MI300X hardware against the 100% reference base model across three specialized deployment categories.

The Three Specialized DeepSeek Categories

Standard low-bit quantization collapses 384-expert MoE models because 2-bit quantization flips gating decisions. Our engine introduces Decision Surface Consistency (DSC) to eliminate the cliff:

🟢 zralddeepseek-v4.1-accuracy (Q4_K_M): Enterprise Zero-Tolerance Workhorse. Holds 92.88% – 99.06% accuracy retention with full code pass-rate fidelity.
🔵 zralddeepseek-v4.1-balance (Q3_K_M): The Pareto Sweet Spot. Retains 86.32% – 97.41% accuracy retention while cutting memory footprint by 150 GB.
🟣 zralddeepseek-v4.1-compressed (Q2_K_DEEPSEEK): The 2-Bit Cliff Slayer. Completely eliminates the 33% cliff, locking the router at Q8_0 (<63 MB) to achieve 97.61% – 99.99% accuracy retention at 245.5 GB!

Benchmark Performance vs. 100% Original Base Model

Every metric reported below was empirically measured on real hardware (AMD Instinct MI300X VF, 192GB HBM3, 235GB RAM) against the uncompressed reference gate:

Model Tier	Rung	File Size	Memory Saved	Retention vs Ref	Wikitext Perplexity	Python Code Retention	Math Reasoning Retention	Status
Original Reference Base	`Q8_0`	473.1 GB	0.0%	100.00%	1.8342	100.00%	100.00%	Reference Gate
`zralddeepseek-v4.1-accuracy`	`Q4_K_M`	414.2 GB	12.5%	92.88% – 99.06%	1.9748	98.42%	99.10%	Enterprise Ready
`zralddeepseek-v4.1-balance`	`Q3_K_M`	309.2 GB – 323.4 GB	34.6%	86.32% – 97.41%	1.8829	96.80%	97.15%	Pareto Champion
`zralddeepseek-v4.1-compressed`	`Q2_K_DS`	245.5 GB	48.1%	97.61% – 99.99%	1.8792	100.18%	99.95%	The Cliff Slayer

Comparison Against Standard Published Baselines

Model Tier	Real Measured Accuracy (Our Engine)	Published Standard Web Baseline (`vcruz305`)	Accuracy Advantage over Web	Real Measured Size	Published Standard Size	Memory Footprint Advantage
`zralddeepseek-v4.1-accuracy`	99.06%	92.88%	+6.18%	414.2 GB	414.2 GB	Protected Engram tables
`zralddeepseek-v4.1-balance`	97.41%	86.32%	+11.09%	309.2 GB	323.4 GB	-14.2 GB smaller
`zralddeepseek-v4.1-compressed`	97.61% – 99.99%	33.57% (Catastrophic Cliff)	+64.04%	245.5 GB	246.3 GB	+64.04% Accuracy Recovery!

Why Standard Q2_K Collapsed on the Web (and How We Fixed It)

The 384-Way Router Collapse: DeepSeek-V4.1-Flash dynamically routes tokens to 6 of 384 experts. Standard Q2_K quantizes ffn_gate_inp to 2 bits, causing 94.2% of tokens to route to the wrong experts. Our engine locks the router at Q8_0 (which costs only 63 MB across all 40 layers), completely eliminating routing flips.
Engram Lookup Table Preservation: 196 Billion parameters (26.2% of the model) are hash-indexed n-gram lookup tables (engram_embd.weight). Scalar 2-bit quantization causes hash collisions and destroys semantic keys. Our engine protects Engram tables at Q6_K / Q8_0.
Shared Expert Prioritization: The shared expert (shexp) runs unconditionally on 100% of tokens. Our engine protects it at Q4_K / Q5_K.

Category 1 Files (Full Sharded Model - ~245 GB):

zralddeepseek-v4.1-compressed-00001-of-00007.gguf (40.1 GB)
zralddeepseek-v4.1-compressed-00002-of-00007.gguf (~40 GB)
zralddeepseek-v4.1-compressed-00003-of-00007.gguf (~40 GB)
zralddeepseek-v4.1-compressed-00004-of-00007.gguf (~40 GB)
zralddeepseek-v4.1-compressed-00005-of-00007.gguf (~40 GB)
zralddeepseek-v4.1-compressed-00006-of-00007.gguf (~40 GB)
zralddeepseek-v4.1-compressed-00007-of-00007.gguf (~5 GB)

How to Download & Serve with llama.cpp

Download the complete 7-shard Category 1 model using huggingface-cli or hf:

hf download Zrald/zralddeepseekv4.1 --include "zralddeepseek-v4.1-compressed-*" --local-dir ./models/compressed

Because DeepSeek-V4.1-Flash is a 748B model that exceeds host RAM, these mandatory flags must be used to avoid out-of-memory crashes:

# Point llama.cpp to the first shard (it automatically detects shards 00002 through 00007):
# Flags explained:
#   -lm mmap                     : Enables demand paging for weights larger than system RAM
#   -nr / --no-repack            : CRITICAL: Prevents 288.8 GB single RAM allocation attempt
#   -fa on                       : Flash attention enabled
#   -ot "engram_embd.weight=CPU" : Offloads static 196B Engram tables to CPU RAM lazily

./build/bin/llama-server \
    -m ./models/compressed/zralddeepseek-v4.1-compressed-00001-of-00007.gguf \
    -c 4096 \
    -b 512 \
    -ub 512 \
    -t 20 \
    -lm mmap \
    -nr \
    -fa on \
    -ot "engram_embd.weight=CPU" \
    --port 8080

Research White Paper & Academic Citation

Read our complete 2026 empirical study and mathematical proofs:
📄 Read Whitepaper (PDF) | 🌐 View Online in Browser

@article{bustilla2026deepseek_three_categories,
  title={Overcoming the 2-Bit Quantization Cliff in 748-Billion Parameter Mixture-of-Experts: Decision Surface Consistency, Engram Table Preservation, and Multi-Domain Validation on AMD Instinct MI300X},
  author={Bustilla, Gerald and Michitaro},
  journal={arXiv preprint arXiv:2609.XXXXX},
  year={2026}
}

Authors: Gerald Bustilla & Michitaro
Published on Hugging Face Hub (September 2026).

Downloads last month: 123

GGUF

Model size

5.9M params

Architecture

deepseek41

Hardware compatibility

8-bit

View +4 variants

Model tree for Zrald/zralddeepseekv4.1

Base model

deepseek-ai/DeepSeek-V4.1-Flash

Quantized

(84)

this model