unable to load gguf in text-generation-webui using llama.cpp however, I can load other gguf models
what gives?
error loading model: unknown model architecture: 'phi2'
llama_load_model_from_file: failed to load model
AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 |
2023-12-18 16:57:06 ERROR:Failed to load the model.
Traceback (most recent call last):
File "/data/text-generation-webui/modules/ui_model_menu.py", line 209, in load_model_wrapper
shared.model, shared.tokenizer = load_model(selected_model, loader)
File "/data/text-generation-webui/modules/models.py", line 89, in load_model
output = load_func_maploader
File "/data/text-generation-webui/modules/models.py", line 259, in llamacpp_loader
model, tokenizer = LlamaCppModel.from_pretrained(model_file)
File "/data/text-generation-webui/modules/llamacpp_model.py", line 91, in from_pretrained
result.model = Llama(**params)
File "/data/llama-cpp-python/llama_cpp/llama.py", line 963, in init
self._n_vocab = self.n_vocab()
File "/data/llama-cpp-python/llama_cpp/llama.py", line 2270, in n_vocab
return self._model.n_vocab()
File "/data/llama-cpp-python/llama_cpp/llama.py", line 252, in n_vocab
assert self.model is not None
AssertionError
I saw a comment about a specific version of llama.cpp
I guess I will hold off for now until it's merged into the main branch (I attempted building from the latest).