Instructions to use GestaltLabs/DeepSeek-V4-Flash-0731-Ornstein with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use GestaltLabs/DeepSeek-V4-Flash-0731-Ornstein with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="GestaltLabs/DeepSeek-V4-Flash-0731-Ornstein")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("GestaltLabs/DeepSeek-V4-Flash-0731-Ornstein")
model = AutoModelForCausalLM.from_pretrained("GestaltLabs/DeepSeek-V4-Flash-0731-Ornstein", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use GestaltLabs/DeepSeek-V4-Flash-0731-Ornstein with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "GestaltLabs/DeepSeek-V4-Flash-0731-Ornstein"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GestaltLabs/DeepSeek-V4-Flash-0731-Ornstein",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/GestaltLabs/DeepSeek-V4-Flash-0731-Ornstein

SGLang

How to use GestaltLabs/DeepSeek-V4-Flash-0731-Ornstein with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "GestaltLabs/DeepSeek-V4-Flash-0731-Ornstein" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GestaltLabs/DeepSeek-V4-Flash-0731-Ornstein",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "GestaltLabs/DeepSeek-V4-Flash-0731-Ornstein" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GestaltLabs/DeepSeek-V4-Flash-0731-Ornstein",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use GestaltLabs/DeepSeek-V4-Flash-0731-Ornstein with Docker Model Runner:
```
docker model run hf.co/GestaltLabs/DeepSeek-V4-Flash-0731-Ornstein
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

DeepSeek-V4-Flash-0731-Ornstein

Release status: complete and verified. All 48 BF16 safetensors shards are published at metadata commit 105ef75053e89b3700f8b6dc793fa7a8c04e12ee, with model.safetensors.index.json and config.json independently verified (36,599 tensors; 608,450,282,488 payload bytes). The companion GGUF release is also complete.

BF16 safetensors for DeepSeek-V4-Flash-0731-Ornstein, a reasoning-oriented fine-tune of deepseek-ai/DeepSeek-V4-Flash-0731 (commit 7872f01b1d1fe23eabc4c98b48bffcef5a386062).

The model architecture is DeepseekV4ForCausalLM (deepseek_v4), a Mixture-of-Experts (MoE) model with 43 layers, hidden size 4096, 256 routed experts (6 active experts per token) plus 1 shared expert, a 128-token sliding window, and a 1,048,576 token context window configured with YaRN RoPE scaling.

The LoRA adapter was trained on Fireworks AI and merged into the base model weights.

Status

This checkpoint injects Ornstein thinking into DeepSeek-V4-Flash-0731. It is an early merge, not a finished quality release. Planned quality work uses RL environments and energy-based fine-tuning.

Evaluation & Verification Limits

Downstream Quality Evaluation

No downstream task evaluation or benchmark suite has been run on this checkpoint yet. Weight-level verification is not evidence of benchmark performance or parity with another model.

Precision & Merge Verification

Merge Integrity: All 33,450 LoRA target linear matrices were promoted to BF16 with adapter updates applied and scale vectors removed. In the pre-export intermediate, 5,499 non-target tensors passed byte-level identity comparisons against the official base release (7872f01b1d1fe23eabc4c98b48bffcef5a386062), and 23 representative numerical probes confirmed BF16-rounding and FP32 tolerance against expected adapter updates.
Publication Verification: All 48 shards were uploaded with per-shard LFS verification, followed by a final metadata commit (105ef75053e89b3700f8b6dc793fa7a8c04e12ee) containing model.safetensors.index.json and config.json. Independent download at that commit confirmed byte-for-byte metadata integrity and index completeness (36,599 tensors).
Dequantization Limit: The official upstream base model is published in mixed FP4/FP8. Dequantizing the remaining non-LoRA FP4/FP8 weights into BF16 removes 2,350 paired scale tensors and produces a predominantly BF16 storage format (preserving 433 F32 tensors and 3 I64 controls), but does not recover precision lost in the official upstream FP4/FP8 quantization.

Support this work

I'm a PhD student in visual neuroscience at the University of Toronto. Training and release compute is self-funded (rented H100s and a local DGX Spark). If these artifacts are useful, Ko-fi helps keep the experiments running.

Model details

Attribute	Specification
Architecture	`DeepseekV4ForCausalLM` (`deepseek_v4`)
Layers	43 hidden layers (3 hash-routing layers, `num_hash_layers: 3`)
Hidden size	4096
Attention	64 query heads, 1 KV head (Multi-Head Latent Attention), `head_dim` 512
Sliding window	128 tokens
Mixture of Experts	256 routed experts (`moe_intermediate_size` 2048), 6 active experts per token, 1 shared expert
Context window	1,048,576 tokens (YaRN RoPE scaling: factor 16, `original_max_position_embeddings` 65536)
Vocabulary size	129,280 tokens
Multi-Token Prediction (MTP) / DSpark	3 physical DSpark stages (`mtp.0`, `mtp.1`, `mtp.2`; target layers [40, 41, 42], `dspark_block_size: 5`, `dspark_markov_rank: 256`; base config records `num_nextn_predict_layers: 1`). All upstream MTP tensors are preserved and dequantized to BF16 (no LoRA applied to MTP)
Target Release Payload	36,599 tensors across 48 shards: 36,163 BF16, 433 preserved F32, 3 preserved I64 controls (608,450,282,488 payload bytes; 2,350 paired UE8M0 scales removed; 24 shards direct, 24 rewritten)
Post-training	PEFT LoRA trained on Fireworks AI; merged across 33,450 linear weight tensors

Usage & Runtime Requirements

Runtime Compatibility Caveats

BF16 safetensors storage does not itself establish runtime compatibility:

Transformers: The observed installed environment release (transformers==4.57.6) lacks support for the DeepseekV4ForCausalLM architecture. Upstream development source (transformers commit cbc1651a, identifying as version 5.17.0.dev0) implements a source-level main trunk mapping with eager attention, but explicitly ignores mtp.* speculative decoding weights and disables assisted generation; full BF16 + DSpark runtime execution has not been verified.
vLLM / SGLang: Upstream vLLM and SGLang recipes were designed for the official FP4/FP8 release backends (such as FlashInfer MXFP4 or DeepGEMM). Loading unscaled BF16 safetensors requires an inference server build that supports unquantized BF16 kernels for DeepSeek-V4 MoE layers.

Chat Encoding & Formatting

The upstream tokenizer does not provide a Jinja chat template. Use the included encoding/encoding_dsv4.py for DeepSeek-V4 prompt formatting and response parsing. Run this example from the downloaded repository directory:

from encoding.encoding_dsv4 import encode_messages

messages = [
    {"role": "user", "content": "Explain the concept of Gestalt grouping principles in visual neuroscience."}
]

# Formats conversation into the required model prompt
prompt = encode_messages(
    messages,
    thinking_mode="thinking",
    reasoning_effort="high"  # Supported: "low", "high", "max"
)

This encoding example and a tokenizer encode/decode round trip were exercised successfully without loading model weights. That is not a full inference test.

Downloading the Model

Download the full repository using the Hugging Face CLI:

# Requires about 610 GB for the files, plus download overhead.
# Full BF16 inference requires a suitable multi-accelerator system.
hf download GestaltLabs/DeepSeek-V4-Flash-0731-Ornstein --local-dir ./DeepSeek-V4-Flash-0731-Ornstein

Files

Path	Description
`model-00001-of-00048.safetensors` … `00048`	Dequantized BF16 model shards (48 shards, ~608.5 GB target payload)
`model.safetensors.index.json`	Shard tensor weight mapping index
`config.json`	Architecture configuration (`DeepseekV4ForCausalLM`)
`generation_config.json`	Generation hyperparameters
`tokenizer.json` / `tokenizer_config.json`	DeepSeek-V4 tokenizer files
`encoding/`	Upstream OpenAI message encoding scripts (`encoding_dsv4.py`)
`LICENSE`	MIT License

License

MIT License, inherited from the upstream DeepSeek-V4-Flash-0731 release.

Downloads last month: 183

Safetensors

Model size

304B params

Tensor type

BF16

F32

I64

Model tree for GestaltLabs/DeepSeek-V4-Flash-0731-Ornstein

Base model

deepseek-ai/DeepSeek-V4-Flash-0731

Finetuned

(33)

this model

Quantizations

1 model

GestaltLabs
/

DeepSeek-V4-Flash-0731-Ornstein