split_special_tokens set to true
Hi Syvai,
Awesome work you guys are doing!!
I have tried out you model. I have been running you model on Runpod using this command:
vllm serve syvai/hviske-v5 --host 0.0.0.0 --port 8006 --trust-remote-code --dtype bfloat16
This didn't work and I finally found out that split_special_tokens was set to true, causing the vLLM's CohereASR decoder prompt to be split into subword garbage by SentencePiece instead of being recognized as their special token IDs. Once I set the split_special_tokens to false, the result was great.
I don't know if it is just because I am running it using vllm cli, but I just wanted to let you know:-)
Hi morten
Thanks for letting us know :) This is just a test release and not the official release - better things are coming!
Looking forward!!