Instructions to use deepseek-ai/DeepSeek-V4.1-Flash with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use deepseek-ai/DeepSeek-V4.1-Flash with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="deepseek-ai/DeepSeek-V4.1-Flash")

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-V4.1-Flash", device_map="auto")

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use deepseek-ai/DeepSeek-V4.1-Flash with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "deepseek-ai/DeepSeek-V4.1-Flash"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4.1-Flash",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/deepseek-ai/DeepSeek-V4.1-Flash

SGLang

How to use deepseek-ai/DeepSeek-V4.1-Flash with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "deepseek-ai/DeepSeek-V4.1-Flash" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4.1-Flash",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "deepseek-ai/DeepSeek-V4.1-Flash" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4.1-Flash",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use deepseek-ai/DeepSeek-V4.1-Flash with Docker Model Runner:
```
docker model run hf.co/deepseek-ai/DeepSeek-V4.1-Flash
```

smaller model with engram?

#32

by ProCreations - opened about 15 hours ago

Discussion

ProCreations

about 15 hours ago

deepseek v4.1 flash is only really “flash” via API, it’s very hard to run locally.
my RTX pro 6000 will thank you if you maybe made a smaller model with ngram???

laterbreh

about 14 hours ago

Or buy more RTX pros. They don't owe you anything.

ProCreations

about 13 hours ago

•

edited about 13 hours ago

Or buy more RTX pros. They don't owe you anything.

did i ever say they owed me a model? am just bringing it up. not like i have infinite money lmao

Bc-AI

about 11 hours ago

@ProCreations just use the chat interface its completely free

ProCreations

about 11 hours ago

@ProCreations just use the chat interface its completely free

completely free... at the cost of my data. API is very cheap but local has its reasons

Bc-AI

about 11 hours ago

@ProCreations oh i see can your GPU run this one? an alternative is Qwen 3.8 27B its not bad.

Bc-AI

about 11 hours ago

@ProCreations this one is not bad too Qwen/Qwen3.8-Flash-Next

ProCreations

about 11 hours ago

@ProCreations this one is not bad too Qwen/Qwen3.8-Flash-Next

i already run this at NVFP4 but quality has been inconsistent and annoying at times

Banaxi-Tech

about 10 hours ago

Deepseek V4.1 Flash Lite 122B would be the best thing I've ever seen for my 96G Mac.

labdhee

about 6 hours ago

I am working on a REAP 50 and a REAP 75 version for SWE specialist, also thinking if can test GSQ and RCO on this .. anyway, within a couple of days finetunes shall start showing up, this model is legit worth having so a little work and investment is well worth it. If i find any success ill share it on here for everyone.

Thanks DS for this commendable contribution to the open source comm.<3

labdhee

about 6 hours ago

deepseek v4.1 flash is only really “flash” via API, it’s very hard to run locally.
my RTX pro 6000 will thank you if you maybe made a smaller model with ngram???

did u try freetoken @ github or something similar.. with your GPU i think as the cache grows throughput shud be faster.. give it a go, try to keep the model on disk and see how much is throughput

ProCreations

about 1 hour ago

deepseek v4.1 flash is only really “flash” via API, it’s very hard to run locally.
my RTX pro 6000 will thank you if you maybe made a smaller model with ngram???

did u try freetoken @ github or something similar.. with your GPU i think as the cache grows throughput shud be faster.. give it a go, try to keep the model on disk and see how much is throughput

Last I checked freetoken doesn’t have support for this model

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment