Thankyou!! this version works with vllm
#1
by
0-hero
- opened
Thankyou!
Awesome!
Great model Migel! You've done it again!
Thank you guys! Enjoy!
migtissera
changed discussion status to
closed
How do you use this with vllm? Can you share a script?
migtissera
changed discussion status to
open
You will need to load the pt files. safetensors and bin for mixtral don't work with vllm as of now