baichuan-inc

/

Baichuan-7B

like 834

Follow

Baichuan Intelligent Technology 272

What is the plan to support transformers >= 4.34.0?

File ".cache/huggingface/modules/transformers_modules/pytorch/tokenization_baichuan.py", line 109, in get_vocab
vocab = {self.convert_ids_to_tokens(i): i for i in range(self.vocab_size)}
File "cache/huggingface/modules/transformers_modules/pytorch/tokenization_baichuan.py", line 105, in vocab_size
return self.sp_model.get_piece_size()
AttributeError: 'BaichuanTokenizer' object has no attribute 'sp_model'