大家百川13b 推理加速是怎么做的？

#23

by yy520xxr - opened Jul 31, 2023

Jul 31, 2023

现在我测了A10大概1s钟 10个汉字左右，有办法加速么？或者并发

Aug 2, 2023

A10跑的int8的版本吗？

Aug 22, 2023

我试过vllm，效果变差了不少，而且不稳定，大家有遇到吗

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment