TheBloke

/

Magicoder-S-DS-6.7B-GGUF

like 75

Hi

Title.

When trying to load using cpp the following err msg appears :

Traceback (most recent call last):
File "D:\text-generation-webui-main\modules\ui_model_menu.py", line 209, in load_model_wrapper
shared.model, shared.tokenizer = load_model(shared.model_name, loader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\text-generation-webui-main\modules\models.py", line 88, in load_model
output = load_func_maploader
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\text-generation-webui-main\modules\models.py", line 253, in llamacpp_loader
model, tokenizer = LlamaCppModel.from_pretrained(model_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\text-generation-webui-main\modules\llamacpp_model.py", line 91, in from_pretrained
result.model = Llama(**params)
^^^^^^^^^^^^^^^
File "D:\text-generation-webui-main\installer_files\env\Lib\site-packages\llama_cpp\llama.py", line 923, in init
self._n_vocab = self.n_vocab()
^^^^^^^^^^^^^^
File "D:\text-generation-webui-main\installer_files\env\Lib\site-packages\llama_cpp\llama.py", line 2184, in n_vocab
return self._model.n_vocab()
^^^^^^^^^^^^^^^^^^^^^
File "D:\text-generation-webui-main\installer_files\env\Lib\site-packages\llama_cpp\llama.py", line 250, in n_vocab
assert self.model is not None
^^^^^^^^^^^^^^^^^^^^^^
AssertionError

Exception ignored in: <function LlamaCppModel.__del__ at 0x0000024AD12E2E80>
Traceback (most recent call last):
File "D:\text-generation-webui-main\modules\llamacpp_model.py", line 49, in del
del self.model