Model cant be loaded
ive downloaded https://github.com/oobabooga/text-generation-webui and https://huggingface.co/anon8231489123/gpt4-x-alpaca-13b-native-4bit-128g and put it in models. When i try to load gpt4-x-alpaca-13b-native-4bit-128g i get an error
Traceback (most recent call last):
File "/home/user/AI/text-generation-webui/installer_files/env/lib/python3.11/site-packages/transformers/modeling_utils.py", line 484, in load_state_dict
return torch.load(checkpoint_file, map_location=map_location)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/AI/text-generation-webui/installer_files/env/lib/python3.11/site-packages/torch/serialization.py", line 986, in load
with _open_file_like(f, 'rb') as opened_file:
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/AI/text-generation-webui/installer_files/env/lib/python3.11/site-packages/torch/serialization.py", line 435, in _open_file_like
return _open_file(name_or_buffer, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/AI/text-generation-webui/installer_files/env/lib/python3.11/site-packages/torch/serialization.py", line 416, in init
super().init(open(name, mode))
^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'models/gpt4-x-alpaca-13b-native-4bit-128g/pytorch_model-00001-of-00006.bin'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user/AI/text-generation-webui/modules/ui_model_menu.py", line 206, in load_model_wrapper
shared.model, shared.tokenizer = load_model(shared.model_name, loader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/AI/text-generation-webui/modules/models.py", line 84, in load_model
output = load_func_maploader
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/AI/text-generation-webui/modules/models.py", line 141, in huggingface_loader
model = LoaderClass.from_pretrained(path_to_model, **params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/AI/text-generation-webui/installer_files/env/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 565, in from_pretrained
return model_class.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/AI/text-generation-webui/installer_files/env/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3307, in from_pretrained
) = cls._load_pretrained_model(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/AI/text-generation-webui/installer_files/env/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3681, in _load_pretrained_model
state_dict = load_state_dict(shard_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/AI/text-generation-webui/installer_files/env/lib/python3.11/site-packages/transformers/modeling_utils.py", line 487, in load_state_dict
with open(checkpoint_file) as f:
^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'models/gpt4-x-alpaca-13b-native-4bit-128g/pytorch_model-00001-of-00006.bin'
I'm having the same issue. Hope someone can chime in with a solution.
I'm trying to run it with CPU (I don't have Nvidia GPU). Also, I am running Oobabooga Text Generation UI inside Pinokio.