Tokenizer class LlamaTokenizer does not exist
Getting this error when I try to load the model using HuggingFace:
ValueError: Tokenizer class LlamaTokenizer does not exist or is not currently imported.
I've updated all of the files to match lmsys/vicuna-13b-delta-v0
Not sure what could be causing this. Are you getting the same error on the delta files?
@xerxes01
Which transformers version are you using?? I get the same error when using transformers version 4.27.4, in the FastChat repo its mentioned that we use transformers with a particular commit id. (Installed using this command "pip3 install git+https://github.com/huggingface/transformers.git@c612628045822f909020f7eb6784c79700813eda")
But i am unable to run inference on the model,
@eachadea
could you please link a colab notebook running the model?
if i use multiple gpus i keep getting TypeError: dispatch_model() got an unexpected keyword argument 'offload_index'
Had the same issue as xerxes01
same here followed the instruction and tried several things floating around even did a second clean setup. still an issue.
The token error should be related to this issue: https://github.com/huggingface/transformers/issues/22222
The token error should be related to this issue: https://github.com/huggingface/transformers/issues/22222
I'm not sure this is related, as that error was regarding LLaMATokenizer not existing (and suggesting to use LlamaTokenizer instead), whereas it looks like OP's error is complaining that LlamaTokenizer does not exist.
I also have this error with another form of Vicuna and updating transformers (or pinning it to previous commits) doesn't seem to have an effect. Stack trace for capitalization example:
(base) $ sudo docker run -p 80:80 -e MODEL=eachadea/vicuna-13b hyperonym/basaran:0.14.1
Downloading (…)okenizer_config.json: 100%|██████████| 727/727 [00:00<00:00, 107kB/s]
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/app/basaran/__main__.py", line 38, in <module>
stream_model = load_model(
File "/app/basaran/model.py", line 318, in load_model
tokenizer = AutoTokenizer.from_pretrained(name_or_path, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/transformers/models/auto/tokenization_auto.py", line 676, in from_pretrained
raise ValueError(
ValueError: Tokenizer class LlamaTokenizer does not exist or is not currently imported.
Did anyone find solution ?
