Hugging Face's logo

deepseek-ai
/
DeepSeek-V2

Text Generation
Transformers
Safetensors
deepseek_v2
conversational
custom_code
text-generation-inference
Inference Endpoints
Model card Files Community
8

How many tokens per second when using Deepseek-V2(236B) as inference model in 8*A100

#7
by harvin-cn - opened
Discussion
harvin-cn
•
edited May 28

I want to deploy in 8*a100,But speed needs considerations,I can't get the inference speed of Deepseek-V2(236B) from README,thanks!

mrcuizhe
•
edited Jul 15

I tried, about 30 tokens/s @harvin-cn

Sign up or log in to comment