Support of Beta API on vLLM
#21
by
taoqiuyang
- opened
We are running DeepSeek-v2.5-236B locally with using vLLM on 8*80GB GPUs as suggested.
It seems the DeepSeek Beta API (with FIM and other new features) are not supported, only v1 API is available.
Is there a way to use DeepSeek Beta API on local GPUs with vLLM or other inferencing libraries?