ggml-vicuna-13b-1.1-q4_3 unrecognized tensor type 5
ggml-vicuna-13b-1.1-q4_1 is working fine for me. However, the q4_3 version seems to have an error. Already updated the llama. I am using the oobabooga-windows.
I'm unsure if ooba uses the latest llama.cpp. If you're certain that it does, then you've updated the files but haven't built with make.
This error is 100% related to you having an older version regardless. Type 5 and type 6 tensors are brand new in llama.cpp.
Ooba doesn't. Grab whl from https://github.com/abetlen/llama-cpp-python/releases/tag/v0.1.36
It is working now, ooba updated his llama. Thank you!
FYI Seems to have reverted again on latest 'master'.
Remove Q4_3 which is no better than Q5 #1218: https://github.com/ggerganov/llama.cpp/pull/1218
Remove Q4_3 which is no better than Q5 #1218: https://github.com/ggerganov/llama.cpp/pull/1218
Thanks. Will update the readme