GPU or CPU
#2
by
lalit34
- opened
Hi Do VLLM models works on CPU because I have tried but its not ?
vllm only works for GPU.
FastChat works for both CPU and GPU https://github.com/lm-sys/FastChat/tree/main#inference-with-command-line-interface
lmzheng
changed discussion status to
closed