Not able to load the model. Getting following errors.

#2
by Nayyer - opened

18:42:18-071089 INFO Loading TheBloke_guanaco-13B-GPTQ
18:42:18-073766 ERROR Failed to load the model.
Traceback (most recent call last):
File "/Users/nayyershahzad/PycharmProjects/textgenwebui/pythonProject/text-generation-webui/modules/ui_model_menu.py", line 213, in load_model_wrapper
shared.model, shared.tokenizer = load_model(selected_model, loader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/nayyershahzad/PycharmProjects/textgenwebui/pythonProject/text-generation-webui/modules/models.py", line 87, in load_model
output = load_func_maploader
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/nayyershahzad/PycharmProjects/textgenwebui/pythonProject/text-generation-webui/modules/models.py", line 366, in GPTQ_loader
import modules.GPTQ_loader
File "/Users/nayyershahzad/PycharmProjects/textgenwebui/pythonProject/text-generation-webui/modules/GPTQ_loader.py", line 9, in
from gptq_for_llama import llama_inference_offload
ModuleNotFoundError: No module named 'gptq_for_llama'

Sign up or log in to comment