Instructions to use PrimeIntellect/DeepSeek-V4-Flash-0731-bf16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use PrimeIntellect/DeepSeek-V4-Flash-0731-bf16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="PrimeIntellect/DeepSeek-V4-Flash-0731-bf16")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("PrimeIntellect/DeepSeek-V4-Flash-0731-bf16")
model = AutoModelForCausalLM.from_pretrained("PrimeIntellect/DeepSeek-V4-Flash-0731-bf16", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use PrimeIntellect/DeepSeek-V4-Flash-0731-bf16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "PrimeIntellect/DeepSeek-V4-Flash-0731-bf16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PrimeIntellect/DeepSeek-V4-Flash-0731-bf16",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/PrimeIntellect/DeepSeek-V4-Flash-0731-bf16

SGLang

How to use PrimeIntellect/DeepSeek-V4-Flash-0731-bf16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "PrimeIntellect/DeepSeek-V4-Flash-0731-bf16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PrimeIntellect/DeepSeek-V4-Flash-0731-bf16",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "PrimeIntellect/DeepSeek-V4-Flash-0731-bf16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PrimeIntellect/DeepSeek-V4-Flash-0731-bf16",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use PrimeIntellect/DeepSeek-V4-Flash-0731-bf16 with Docker Model Runner:
```
docker model run hf.co/PrimeIntellect/DeepSeek-V4-Flash-0731-bf16
```

DeepSeek-V4-Flash-0731-bf16

A bfloat16 re-export of deepseek-ai/DeepSeek-V4-Flash-0731. The weights are the published ones, dequantized: the dense linears from fp8 (e4m3 with ue8m0 scales over 128x128 blocks) and the routed experts from MXFP4, both to plain bfloat16. 43 layers, 114 safetensors shards.

Why this exists

The upstream config.json carries quantization_config and expert_dtype = "fp4". An inference engine reading those fields allocates fp8 linear parameters and MXFP4 expert parameters, which bfloat16 weights cannot fill. The routed experts at least fail loudly, in vLLM's _load_w13, where the packed MXFP4 shape does not match. The fp8 linears fail silently: a bfloat16 tensor has the same shape as the fp8 parameter it is loaded into, so copy_ casts it elementwise and no dequantization scale is ever applied. Serving bfloat16 weights therefore requires a checkpoint whose config carries neither field, which is what this repo is.

Differences from upstream

quantization_config and expert_dtype removed from config.json.
num_nextn_predict_layers, dspark_block_size, dspark_markov_rank, dspark_noise_token_id and dspark_target_layer_ids removed from config.json.
No mtp.* weights (upstream ships 4,705 of them; this export has none), so speculative decoding, including DSpark, is not available here. Do not pass --speculative-config or --speculative-algorithm DSPARK.
Not everything is bfloat16: the mHC, attention-sink and bias tensors are kept in fp32, and the router hash table in int64. The export holds 33,814 bf16 tensors, 406 fp32 tensors and 3 int64 tensors.

Prompting

There is no Jinja chat template, by upstream design. Use the bundled encoding/ directory, which is copied verbatim from upstream and is the reference implementation for turning OpenAI-format messages into an input string and for parsing completions back.

prime-rl users: MODEL_RENDERER_MAP is keyed on the upstream repo id (deepseek-ai/DeepSeek-V4-Flash-0731), so this repo id does not resolve automatically and falls back to the default renderer. Set the renderer explicitly:

[orchestrator.renderer]
name = "deepseek-v4"

Provenance

Exported from the conversion cache that prime-rl's trainer writes the first time it loads the published checkpoint, which is where the fp8 and MXFP4 weights are dequantized to bfloat16. The metadata.total_size in model.safetensors.index.json is byte-identical to that cache's (568,753,728,604 bytes), so the cache-to-HF round trip drops nothing.

License

MIT, inherited from upstream. The upstream LICENSE is included verbatim.

Downloads last month: 19,788

Safetensors

Model size

284B params

Tensor type

F32

BF16

I64

Model tree for PrimeIntellect/DeepSeek-V4-Flash-0731-bf16

Base model

deepseek-ai/DeepSeek-V4-Flash-0731

Finetuned

(38)

this model