keeps crashing when running on text-generation-webui
#13
by
wesleysanjose
- opened
tried to run this GPTQ using autoGPTQ in text-generation-webui, but it keeps crashing after interfering for seconds. i can see GPU spikes but the temperature was not crazy. the non GPTQ version runs fine. I am using 3090 BTW.
Same issue same card. Using ooba webui on commit 63ece46
ooba-webui | 2023-07-21 18:47:51 ERROR:Failed to load the model.
ooba-webui | Traceback (most recent call last):
ooba-webui | File "/app/server.py", line 68, in load_model_wrapper
ooba-webui | shared.model, shared.tokenizer = load_model(shared.model_name, loader)
ooba-webui | File "/app/modules/models.py", line 87, in load_model
ooba-webui | tokenizer = load_tokenizer(model_name, model)
ooba-webui | File "/app/modules/models.py", line 104, in load_tokenizer
ooba-webui | tokenizer = AutoTokenizer.from_pretrained(
ooba-webui | File "/app/venv/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 702, in from_pretrained
ooba-webui | return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
ooba-webui | File "/app/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1841, in from_pretrained
ooba-webui | return cls._from_pretrained(
ooba-webui | File "/app/venv/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2004, in _from_pretrained
ooba-webui | tokenizer = cls(*init_inputs, **init_kwargs)
ooba-webui | File "/app/venv/lib/python3.10/site-packages/transformers/models/gpt2/tokenization_gpt2.py", line 194, in __init__
ooba-webui | with open(merges_file, encoding="utf-8") as merges_handle:
ooba-webui | TypeError: expected str, bytes or os.PathLike object, not NoneType