Instructions to use deepseek-ai/DeepSeek-V4.1-Flash with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use deepseek-ai/DeepSeek-V4.1-Flash with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="deepseek-ai/DeepSeek-V4.1-Flash")

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-V4.1-Flash", device_map="auto")

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use deepseek-ai/DeepSeek-V4.1-Flash with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "deepseek-ai/DeepSeek-V4.1-Flash"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4.1-Flash",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/deepseek-ai/DeepSeek-V4.1-Flash

SGLang

How to use deepseek-ai/DeepSeek-V4.1-Flash with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "deepseek-ai/DeepSeek-V4.1-Flash" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4.1-Flash",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "deepseek-ai/DeepSeek-V4.1-Flash" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deepseek-ai/DeepSeek-V4.1-Flash",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use deepseek-ai/DeepSeek-V4.1-Flash with Docker Model Runner:
```
docker model run hf.co/deepseek-ai/DeepSeek-V4.1-Flash
```

Lowest-latency hosted provider for DeepSeek V4.1 Flash from Northern Virginia (US East)?

#57

by KekeJ - opened 7 days ago

Discussion

KekeJ

7 days ago

Hi DeepSeek team and community,

Which hosted provider currently offers the lowest time to first token (TTFT) and end-to-end response latency for DeepSeek-V4.1-Flash, with requests originating from Northern Virginia, US East?

I am looking for low-latency text-only inference with short text/JSON responses and thinking disabled where supported. I am interested in this exact V4.1 model, rather than V4-Flash-0731 or a smaller/distilled variant.

I checked the existing discussions and the Hugging Face provider comparison. On September 16, 2026, the page showed:

Provider	Listed latency	Listed throughput
Baseten	0.43 s	182 tokens/s
DeepInfra	0.54 s	14 tokens/s
Fireworks	0.60 s	86 tokens/s
Novita	0.94 s	72 tokens/s

These are the public HF comparison figures, not my own Virginia measurements. The page does not establish that the measurements originate in Northern Virginia, so I do not want to assume that its ranking applies to that location or to short-output requests.

Does anyone have reproducible measurements or deployment recommendations covering:

Client-observed TTFT and total response latency from Northern Virginia, ideally p50/p95/p99, with sample count and test date.
The exact provider/model ID, serving region, and whether the requests used the provider directly, Hugging Face, or another gateway.
Input/output token counts, reasoning settings, concurrency, connection reuse, and cold vs. warm/prefix-cached conditions.
A regional endpoint or region-pinning option in/near Virginia, and whether it applies to serverless or requires a dedicated deployment.
Current input/output/cache pricing, or the minimum cost and capacity for a dedicated low-latency deployment.

Baseten looks like a promising candidate from the HF figures. Has anyone compared it with Fireworks, DeepInfra, Novita, or another provider from US East using the same short-output workload?

I am particularly interested in TTFT and tail latency, not just peak generation tokens per second. Providers are welcome to reply; please identify your affiliation and distinguish measured results from estimates.

Thanks!

Avifenesh

5 days ago

I host DeepSeek V4.1 Flash (and GLM-5.3-Flash) at https://inference.tiyuvta.ai/app - OpenAI-compatible. DSV4.1 Flash is $0.30/1M in, $1.20 out, $0.006 cache.

Fair warning: my boxes are not in US East, so if Virginia TTFT is the hard requirement I would not beat a NoVA-colocated provider on that axis. Happy to answer anything else about the endpoint.

davehh

1 day ago

4 sparks will get you 40 tok/s ..

CoderX1997

1 day ago

Hi DeepSeek team and community,

Which hosted provider currently offers the lowest time to first token (TTFT) and end-to-end response latency for DeepSeek-V4.1-Flash, with requests originating from Northern Virginia, US East?

I am looking for low-latency text-only inference with short text/JSON responses and thinking disabled where supported. I am interested in this exact V4.1 model, rather than V4-Flash-0731 or a smaller/distilled variant.

I checked the existing discussions and the Hugging Face provider comparison. On September 16, 2026, the page showed:

Provider Listed latency Listed throughput

Baseten 0.43 s 182 tokens/s

DeepInfra 0.54 s 14 tokens/s

Fireworks 0.60 s 86 tokens/s

Novita 0.94 s 72 tokens/s

These are the public HF comparison figures, not my own Virginia measurements. The page does not establish that the measurements originate in Northern Virginia, so I do not want to assume that its ranking applies to that location or to short-output requests.

Does anyone have reproducible measurements or deployment recommendations covering:

Client-observed TTFT and total response latency from Northern Virginia, ideally p50/p95/p99, with sample count and test date.

The exact provider/model ID, serving region, and whether the requests used the provider directly, Hugging Face, or another gateway.

Input/output token counts, reasoning settings, concurrency, connection reuse, and cold vs. warm/prefix-cached conditions.

A regional endpoint or region-pinning option in/near Virginia, and whether it applies to serverless or requires a dedicated deployment.

Current input/output/cache pricing, or the minimum cost and capacity for a dedicated low-latency deployment.

Baseten looks like a promising candidate from the HF figures. Has anyone compared it with Fireworks, DeepInfra, Novita, or another provider from US East using the same short-output workload?

I am particularly interested in TTFT and tail latency, not just peak generation tokens per second. Providers are welcome to reply; please identify your affiliation and distinguish measured results from estimates.

Thanks!

Consider looking at all the providers on Open router. If none of them work consider just spending a bit of money on DGX sparks / GPUs.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment