bartowski

/

Nous-Capybara-34B-exl2

like 5

Using Text Gen Webui and exllama2 0.0.8 wheels :

The model loads correctly with exllama2 as loader but with exllama2_HF I get the following error :

2023-11-15 17:22:43 INFO:Loading bartowski_Nous-Capybara-34B-exl2_4.25... [exl2_hf loader]
2023-11-15 17:23:38 ERROR:Failed to load the model.
Traceback (most recent call last):
  File "L:\...\text-generation-webui\modules\ui_model_menu.py", line 210, in load_model_wrapper
    shared.model, shared.tokenizer = load_model(shared.model_name, loader)
  File "L:\...\text-generation-webui\modules\models.py", line 93, in load_model
    tokenizer = load_tokenizer(model_name, model)
  File "L:\...\text-generation-webui\modules\models.py", line 113, in load_tokenizer
    tokenizer = AutoTokenizer.from_pretrained(
  File "L:\...\text-generation-webui\installer_files\env\lib\site-packages\transformers\models\auto\tokenization_auto.py", line 751, in from_pretrained
    tokenizer_class = get_class_from_dynamic_module(class_ref, pretrained_model_name_or_path, **kwargs)
  File "L:\...\text-generation-webui\installer_files\env\lib\site-packages\transformers\dynamic_module_utils.py", line 499, in get_class_from_dynamic_module
    return get_class_in_module(class_name, final_module.replace(".py", ""))
  File "L:\...\text-generation-webui\installer_files\env\lib\site-packages\transformers\dynamic_module_utils.py", line 199, in get_class_in_module
    module = importlib.import_module(module_path)
  File "L:\...\text-generation-webui\installer_files\env\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'transformers_modules.bartowski_Nous-Capybara-34B-exl2_4'

2023-11-15 17:25:37 INFO:Loading bartowski_Nous-Capybara-34B-exl2_4.25... [exl2 loader]
2023-11-15 17:26:33 INFO:Loaded the model in 56.14 seconds.
2023-11-15 17:31:02 INFO:Loading bartowski_airoboros-2.2.1-y34b-exl2_4.25...   [exl2 loader]
2023-11-15 17:31:55 INFO:Loaded the model in 52.89 seconds.
2023-11-15 17:32:05 INFO:Loading bartowski_airoboros-2.2.1-y34b-exl2_4.25...  [exl2_hf loader]
2023-11-15 17:32:58 INFO:Loaded the model in 52.72 seconds.
2023-11-15 17:34:07 INFO:Loading bartowski_Nous-Capybara-34B-exl2_4_25...  [exl2_hf loader]
2023-11-15 17:35:00 INFO:Loaded the model in 53.17 seconds.

Fixed by :

Changing directory name to transformers_modules.bartowski_Nous-Capybara-34B-exl2_4_25

Notes :

I didn't meet this issue with your Airo-YI:4.25 repo, making this issue quite mystic from my perspective (not enough free time to dig into it right now).

Conclusion (good practice) :

Avoiding the use of dots in branch names.

Thanks as always for the amazing work 🤗