TGI Support

#23
by iz2057 - opened

Is anyone able to get this working on text-generation-inference? I'm getting the following errors:

The tokenizer class you load from this checkpoint is 'PreTrainedTokenizerFast'. 
The class this function is called from is 'Qwen2Tokenizer'.
File "/opt/conda/lib/python3.10/site-packages/transformers/models/qwen2/tokenization_qwen2.py", line 172, in __init__
    with open(vocab_file, encoding="utf-8") as vocab_handle:

TypeError: expected str, bytes or os.PathLike object, not NoneType

It is working with latest text-generation-webui and we are using it. Have you tried with latest TGI version ?

Sign up or log in to comment