It currently gives an error when you try to inference it
Hi @mlinmg Thanks for the issue, unfortunately this repo is not intended for VLLM (yet). I am using transformers + https://github.com/huggingface/transformers/pull/27950
· Sign up or log in to comment
