internlm

/

internlm-chat-7b

like 98

(textgen) [root@pve-m7330 text-generation-webui]# python server.py --api --listen --trust-remote-code --disk-cache-dir /data/tmp --use_double_quant --quant_type nf4 --numa --load-in-4bit --settings settings-template.yaml --model models/internlm-chat-7b/
2023-09-30 13:38:28 WARNING:trust_remote_code is enabled. This is dangerous.
2023-09-30 13:38:28 WARNING:
You are potentially exposing the web UI to the entire internet without any access password.
You can create one with the "--gradio-auth" flag like this:

--gradio-auth username:password

Make sure to replace username:password with your own.
2023-09-30 13:38:31 INFO:Loading settings from settings-template.yaml...
2023-09-30 13:38:31 INFO:Loading internlm-chat-7b...
2023-09-30 13:38:31 INFO:Using the following 4-bit params: {'load_in_4bit': True, 'bnb_4bit_compute_dtype': torch.float16, 'bnb_4bit_quant_type': 'nf4', 'bnb_4bit_use_double_quant': True}
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:07<00:00, 1.03it/s]
Traceback (most recent call last):
File "/home/user/text-generation-webui/server.py", line 222, in
shared.model, shared.tokenizer = load_model(model_name)
File "/home/user/text-generation-webui/modules/models.py", line 86, in load_model
tokenizer = load_tokenizer(model_name, model)
File "/home/user/text-generation-webui/modules/models.py", line 105, in load_tokenizer
tokenizer = AutoTokenizer.from_pretrained(
File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 738, in from_pretrained
return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2042, in from_pretrained
return cls._from_pretrained(
File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2253, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
File "/root/.cache/huggingface/modules/transformers_modules/internlm-chat-7b/tokenization_internlm.py", line 68, in init
super().init(
File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/tokenization_utils.py", line 366, in init
self._add_tokens(self.all_special_tokens_extended, special_tokens=True)
File "/home/user/miniconda3/envs/textgen/lib/python3.10/site-packages/transformers/tokenization_utils.py", line 454, in _add_tokens
current_vocab = self.get_vocab().copy()
File "/root/.cache/huggingface/modules/transformers_modules/internlm-chat-7b/tokenization_internlm.py", line 108, in get_vocab
vocab = {self.convert_ids_to_tokens(i): i for i in range(self.vocab_size)}
File "/root/.cache/huggingface/modules/transformers_modules/internlm-chat-7b/tokenization_internlm.py", line 96, in vocab_size
return self.sp_model.get_piece_size()
AttributeError: 'InternLMTokenizer' object has no attribute 'sp_model'