Cant run the model
Hey guys, this might be a noob question but Im struggling to run this model. getting the following error
Exception has occurred: AttributeError
'ReplitLMTokenizer' object has no attribute 'sp_model'
File "/home/../.cache/huggingface/modules/transformers_modules/glaiveai/glaive-function-calling-v2-small/ee756352b27602951abdc7618c9139f1bb8e5031/replit_lm_tokenizer.py", line 85, in vocab_size
return self.sp_model.get_piece_size()
File "/home/../.cache/huggingface/modules/transformers_modules/glaiveai/glaive-function-calling-v2-small/ee756352b27602951abdc7618c9139f1bb8e5031/replit_lm_tokenizer.py", line 88, in get_vocab
vocab = {self.convert_ids_to_tokens(i): i for i in range(self.vocab_size)}
File "/home/../.cache/huggingface/modules/transformers_modules/glaiveai/glaive-function-calling-v2-small/ee756352b27602951abdc7618c9139f1bb8e5031/replit_lm_tokenizer.py", line 78, in init
super().init(bos_token=bos_token, eos_token=eos_token, unk_token=unk_token, pad_token=pad_token, sep_token=sep_token, sp_model_kwargs=self.sp_model_kwargs, **kwargs)
File "/LocalStorage/ScratchPad/vllmtest.py", line 14, in
llm = LLM(model="glaiveai/glaive-function-calling-v2-small",trust_remote_code=True)
AttributeError: 'ReplitLMTokenizer' object has no attribute 'sp_model'
I got to the bottom of this, it seems like the recent update to replit tokenizer has something messed up.
You need to swap some lines around in def init
You need to set self.sp_model before calling super().init
Also there is some lines in hf_prefixlm that needed commenting out.
Adjust the lines of replit_lm_tokenizer after line 64 to look like this
self.sp_model_kwargs = {} if sp_model_kwargs is None else sp_model_kwargs
self.vocab_file = vocab_file
self.sp_model = spm.SentencePieceProcessor(**self.sp_model_kwargs)
self.sp_model.Load(vocab_file)
super().init(bos_token=bos_token, eos_token=eos_token, unk_token=unk_token, pad_token=pad_token, sep_token=sep_token, sp_model_kwargs=self.sp_model_kwargs, **kwargs)