DeepSeek-V4-Flash-0731-AFM-MLX

Instructions to use maclocal/DeepSeek-V4-Flash-0731-AFM-MLX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use maclocal/DeepSeek-V4-Flash-0731-AFM-MLX with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="maclocal/DeepSeek-V4-Flash-0731-AFM-MLX")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("maclocal/DeepSeek-V4-Flash-0731-AFM-MLX")
model = AutoModelForCausalLM.from_pretrained("maclocal/DeepSeek-V4-Flash-0731-AFM-MLX", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use maclocal/DeepSeek-V4-Flash-0731-AFM-MLX with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "maclocal/DeepSeek-V4-Flash-0731-AFM-MLX"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "maclocal/DeepSeek-V4-Flash-0731-AFM-MLX",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/maclocal/DeepSeek-V4-Flash-0731-AFM-MLX

SGLang

How to use maclocal/DeepSeek-V4-Flash-0731-AFM-MLX with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "maclocal/DeepSeek-V4-Flash-0731-AFM-MLX" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "maclocal/DeepSeek-V4-Flash-0731-AFM-MLX",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "maclocal/DeepSeek-V4-Flash-0731-AFM-MLX" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "maclocal/DeepSeek-V4-Flash-0731-AFM-MLX",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use maclocal/DeepSeek-V4-Flash-0731-AFM-MLX with Docker Model Runner:
```
docker model run hf.co/maclocal/DeepSeek-V4-Flash-0731-AFM-MLX
```

Experimental AFM artifact. This repository is intended for a later version of maclocal-api that is still in development and has not yet been released. It may not work with current public releases or other runtimes, and its behavior, quality, performance, memory use, compatibility, or file layout may change. For stable general-purpose MLX models, use mlx-community instead.

DeepSeek-V4-Flash-0731

AFM native MLX conversion

This repository contains an AFM-native MLX conversion of the official deepseek-ai/DeepSeek-V4-Flash-0731 checkpoint. It was produced with AFM's built-in, shard-streaming converter and uses conversion format version 11 with the native profile.

The checkpoint preserves the model's embedded DSpARK/MTP support tensors. Its mixed quantization layout uses MXFP4 (4-bit, group size 32) for routed experts and primarily MXFP8 (8-bit, group size 32) for attention, shared projections, and MTP support projections. The authoritative per-tensor mapping is recorded in config.json, while .afm-mlx-conversion.json records conversion progress and output-file provenance.

The equivalent local conversion command is:

afm mlx-convert \
  --source /path/to/DeepSeek-V4-Flash-0731 \
  --output /path/to/DeepSeek-V4-Flash-0731-AFM-MLX

Run the converted checkpoint through AFM's native MLX provider:

afm mlx \
  --model scouzi1966/DeepSeek-V4-Flash-0731-AFM-MLX \
  --no-think

Enable the checkpoint's embedded DSpARK/MTP speculative decoder with:

afm mlx \
  --model scouzi1966/DeepSeek-V4-Flash-0731-AFM-MLX \
  --mtp \
  --no-think

This is an AFM deployment checkpoint, not a generic Transformers conversion. Use an AFM release containing native deepseek_v4 and embedded DSpARK support.

Technical Report👁️

Introduction

DeepSeek-V4-Flash-0731 is the official release of DeepSeek-V4-Flash, superseding the preview version, with substantially enhanced agentic capabilities. It has the same model structure as DeepSeek-V4-Flash-DSpark, i.e. it comes with a speculative decoding module attached.

DeepSeek-V4-Flash-0731 outperforms DeepSeek-V4-Pro (Preview) on benchmarks listed below despite its far smaller activated parameter count, and is broadly competitive with the strongest proprietary models available.

Benchmark	DeepSeek-V4-Flash-0731	DeepSeek-V4-Flash (Preview)	DeepSeek-V4-Pro (Preview)	GLM-5.2	Opus-4.8
Terminal Bench 2.1	82.7	61.8	72.1	81.0	85.0
NL2Repo	54.2	39.4	38.5	48.9	69.7
Cybergym	76.7	38.7	52.7	-	83.1
DeepSWE	54.4	7.3	12.8	46.2	58.0
Toolathlon-Verified	70.3	49.7	55.9	59.9	76.2
Agents' Last Exam	25.2	15.8	16.5	23.8	25.7
AutomationBench Public	25.1	10.8	12.8	12.9	27.2
DSBench-FullStack †	68.7	37.0	41.8	61.8	71.6
DSBench-Hard †	59.6	25.8	31.1	54.5	71.7

Notes:

For the Code Agent tasks among the public benchmarks above, DeepSeek-V4-Flash-0731 is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework, using the max reasoning effort level with temperature = 1.0, top_p = 0.95.
† DSBench-FullStack is an internal full-stack development test set; DSBench-Hard is an internal test set of difficult coding-agent problems.

Chat Template

This release does not include a Jinja-format chat template. Instead, we provide a dedicated encoding folder with Python scripts and test cases demonstrating how to encode messages in OpenAI-compatible format into input strings for the model, and how to parse the model's text output. Please refer to the encoding folder for full documentation.

The reasoning_effort parameter now supports three levels — low, high, and max — which control how much deliberation the model spends before answering.

A brief example:

from encoding_dsv4 import encode_messages, parse_message_from_completion_text

messages = [
    {"role": "user", "content": "hello"},
    {"role": "assistant", "content": "Hello! I am DeepSeek.", "reasoning_content": "thinking..."},
    {"role": "user", "content": "1+1=?"}
]

# messages -> string
prompt = encode_messages(messages, thinking_mode="thinking", reasoning_effort="max")

# string -> tokens
import transformers
tokenizer = transformers.AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-V4-Flash-0731")
tokens = tokenizer.encode(prompt)

How to Run with vLLM

DSpark speculative decoding is enabled with a single flag — add --speculative-config with method: dspark to your vLLM launch command:

--speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"greedy"}'

For example, the command below serves the model with vLLM on a single 4×GB300 node. See the vLLM recipe for detailed instructions and other hardware configurations.

vllm serve deepseek-ai/DeepSeek-V4-Flash-0731 \
  --trust-remote-code --kv-cache-dtype fp8 --block-size 256 \
  --data-parallel-size 4 --enable-expert-parallel \
  --moe-backend deep_gemm_mega_moe \
  --attention-config '{"use_fp4_indexer_cache": true}' \
  --speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"greedy"}'

How to Run with SGLang

Enable DSpark with --speculative-algorithm DSPARK and do not set a separate --speculative-draft-model-path as the target and draft weights therefore come from the same checkpoint. See the SGLang cookbook for detailed instructions, benchmarks and other hardwares configurations.

sglang serve \
  --trust-remote-code \
  --model-path deepseek-ai/DeepSeek-V4-Flash-0731 \
  --tp 4 \
  --moe-runner-backend flashinfer_mxfp4 \
  --speculative-algorithm DSPARK \
  --mem-fraction-static 0.90 \
  --chunked-prefill-size 4096 \
  --swa-full-tokens-ratio 0.1 \

How to Run Locally

Please refer to the inference folder for detailed instructions on running DeepSeek-V4 locally, including model weight conversion and interactive chat demos.

For local deployment, we recommend setting the sampling parameters to temperature = 1.0, with top_p = 0.95 for agentic scenarios and top_p = 1.0 otherwise. For the high and max reasoning effort levels, we recommend a maximum output length of 384K tokens.

License

This repository and the model weights are licensed under the MIT License.

Citation

@misc{deepseekai2026deepseekv4,
      title={DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence},
      author={DeepSeek-AI},
      year={2026},
}

Contact

If you have any questions, please raise an issue or contact us at service@deepseek.com.

Downloads last month: -

Safetensors

Model size

304B params

Tensor type

BF16

F32

I64

U32

Paper for maclocal/DeepSeek-V4-Flash-0731-AFM-MLX

DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence

Paper • 2606.19348 • Published Apr 26 • 43