Won't load on text-generation-webui
When I try loading saily_220b.Q3_K_M.gguf using text-generation-webui I got error:
Traceback (most recent call last):
File "S:\text-generation-webui\modules\ui_model_menu.py", line 214, in load_model_wrapper
shared.model, shared.tokenizer = load_model(selected_model, loader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "S:\text-generation-webui\modules\models.py", line 90, in load_model
output = load_func_maploader
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "S:\text-generation-webui\modules\models.py", line 260, in llamacpp_loader
model, tokenizer = LlamaCppModel.from_pretrained(model_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "S:\text-generation-webui\modules\llamacpp_model.py", line 101, in from_pretrained
result.model = Llama(**params)
^^^^^^^^^^^^^^^
File "S:\text-generation-webui\installer_files\env\Lib\site-packages\llama_cpp_cuda\llama.py", line 932, in init
self._ctx = _LlamaContext(
^^^^^^^^^^^^^^
File "S:\text-generation-webui\installer_files\env\Lib\site-packages\llama_cpp_cuda\llama.py", line 420, in init
self.ctx = llama_cpp.llama_new_context_with_model(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "S:\text-generation-webui\installer_files\env\Lib\site-packages\llama_cpp_cuda\llama_cpp.py", line 667, in llama_new_context_with_model
return _lib.llama_new_context_with_model(model, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: exception: access violation reading 0x0000000000000000