ValueError: Tokenizer class BitnetTokenizer does not exist or is not currently imported.
Updated transformers library with pip install --upgrade transformers, but still receive following error:
ValueError Traceback (most recent call last)
in <cell line: 3>()
1 from transformers import AutoTokenizer, AutoModelForCausalLM
2
----> 3 tokenizer = AutoTokenizer.from_pretrained("1bitLLM/bitnet_b1_58-3B", trust_remote_code=True)
4 model = AutoModelForCausalLM.from_pretrained("1bitLLM/bitnet_b1_58-3B")
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/tokenization_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *inputs, **kwargs)
839 tokenizer_class = tokenizer_class_from_name(tokenizer_class_candidate)
840 if tokenizer_class is None:
--> 841 raise ValueError(
842 f"Tokenizer class {tokenizer_class_candidate} does not exist or is not currently imported."
843 )
ValueError: Tokenizer class BitnetTokenizer does not exist or is not currently imported.