audreyt

/

Taiwan-LLaMa-v1.0-GGUF

like 13

Using https://github.com/oobabooga/text-generation-webui with https://github.com/oobabooga/text-generation-webui/pull/3877 and GGUF files from this repository, I get the exception

2023-09-13 05:58:28 INFO:Using ctransformers model_type: llama for models/Taiwan-LLaMa-13b-1.0.Q8_0.gguf
2023-09-13 05:58:28 INFO:Loaded the model in 6.09 seconds.

2023-09-13 05:58:28 ERROR:Failed to load the model.
Traceback (most recent call last):
  File "/app/modules/ui_model_menu.py", line 199, in load_model_wrapper
    settings = get_model_metadata(selected_model)
  File "/app/modules/models_settings.py", line 49, in get_model_metadata
    metadata = metadata_gguf.load_metadata(model_file)
  File "/app/modules/metadata_gguf.py", line 75, in load_metadata
    raise Exception('You are using an outdated GGUF, please download a new one.')
Exception: You are using an outdated GGUF, please download a new one.

It seems as though the model is using GGUFv1 while other GGUF models use GGUFv2
https://github.com/oobabooga/text-generation-webui/issues/3880#issuecomment-1715913153

Could these models be upgraded to use v2 as well?