OSError: Error no file named pytorch_model.bin, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory models\TheBlokestable-vicuna-13B-GPTQ.
First attempted to download via but get the following error:
Traceback (most recent call last):
File “...\AIchatbot\oogabooga\oobabooga-windows\text-generation-webui\server.py”, line 172, in download_model_wrapper
links, sha256, is_lora = downloader.get_download_links_from_huggingface(model, branch, text_only=False)
File “...\AIchatbot\oogabooga\oobabooga-windows\text-generation-webui\download-model.py”, line 103, in get_download_links_from_huggingface
fname = dict[i][‘path’]
KeyError: 0
So I downloaded manually instead. Added the model in models/TheBloke/stable-vicuna-13B-GPTQ
I now get this error:
Traceback (most recent call last): File “...\AIchatbot\oogabooga\oobabooga-windows\text-generation-webui\server.py”, line 85, in load_model_wrapper shared.model, shared.tokenizer = load_model(shared.model_name) File “...\AIchatbot\oogabooga\oobabooga-windows\text-generation-webui\modules\models.py”, line 175, in load_model model = LoaderClass.from_pretrained(checkpoint, **params) File “...\AIchatbot\oogabooga\oobabooga-windows\installer_files\env\lib\site-packages\transformers\models\auto\auto_factory.py”, line 471, in from_pretrained return model_class.from_pretrained( File “...\AIchatbot\oogabooga\oobabooga-windows\installer_files\env\lib\site-packages\transformers\modeling_utils.py”, line 2405, in from_pretrained raise EnvironmentError( OSError: Error no file named pytorch_model.bin, tf_model.h5, model.ckpt.index or flax_model.msgpack found in directory models\TheBloke.
I have also tried ignoring this error and entering the GPTQ parameters Bits = 4, Groupsize 128, Llama
however this does not change the error I receive.
Thanks
Note, I am intending to run via GPU on a 4090. Additionally other models do work.
Please read the README. You missed steps 5 - 9:
Please read the README. You missed steps 5 - 9:
Thanks for the reply, but I did not miss steps 5 - 9.
I'm happy to provide any information that may help, but I repeated steps 5 - 9 multiple times with no luck.
OK please show me the contents of your models folder and the contents of the model folder (which I think is calledmodels\TheBloke ? based on the errror message) .
If you've called the model folder TheBloke then the model files need to be directly in that folder.
Eg if you had models/TheBloke/stable-vicuna-13B-GPTQ/<model files here> that would not work.
Same issue here, multiple reinstalls:
\text-generation-webui\models\TheBloke_stable-vicuna-13B-GPTQ
Models:
Someone on Discord recently told me this :
So please try that advice: go to the Models screen, manually set wbits = 4, groupsize = 128, model_type = llama, then reload model. Does that work?
If it does, please try clicking "Save settings for this model", then restart the UI and see if it works then. If it does, then you're fine (and didn't follow the README!). If it doesn't, then you're hitting this same issue, which must be a bug in text-generation-webui.
Same error, coming from a fresh oogabooga install
I have the model + other files directly in this path:
...\AIchatbot\oogabooga\oobabooga-windows\text-generation-webui\models\TheBloke
I also have this on my start-webui.bat:call python server.py --auto-devices --chat --wbits 4 --gpu-memory 22 --groupsize 128
Which potentially could have been causing the issue, though the settings are technically correct.
The only difference I did this time was set the settings before choosing the model. After choosing the model it reset the settings again, but I chose them again and saved.
I don't think choosing the settings before the model made any difference but somehow I got it working now.
make a .json file in your TheBloke_stable-vicuna-13B-GPTQ named pytorch_model.bin.index with this text
{
"metadata": {
"total_size": 26031759360
},
"weight_map": {
"lm_head.weight": "pytorch_model-00003-of-00003.bin",
"model.embed_tokens.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.0.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.0.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.0.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.0.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.0.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.0.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
"model.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.1.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.1.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.1.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.1.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.1.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.1.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.1.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.1.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.1.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
"model.layers.1.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.10.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.10.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.10.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.10.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.10.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.10.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.10.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.10.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.10.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
"model.layers.10.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.11.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.11.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.11.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.11.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.11.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.11.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.11.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.11.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.11.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
"model.layers.11.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.12.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.12.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.12.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.12.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.12.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.12.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.12.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.12.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.12.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
"model.layers.12.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.13.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.13.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.13.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.13.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.13.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.13.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.13.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.13.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.13.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
"model.layers.13.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.14.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.14.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.14.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.14.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.14.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.14.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.14.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.14.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.14.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
"model.layers.14.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.15.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.15.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.15.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.15.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.15.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.15.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.15.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.15.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.15.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
"model.layers.15.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.16.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.16.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.16.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.16.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.16.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.16.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.16.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.16.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.16.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
"model.layers.16.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.17.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.17.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.17.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.17.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.17.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.17.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.17.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.17.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.17.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
"model.layers.17.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.18.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.18.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.18.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.18.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.18.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.18.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.18.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.18.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.18.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
"model.layers.18.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.19.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.19.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.19.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.19.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.19.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.19.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.19.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.19.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.19.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
"model.layers.19.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.2.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.2.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.2.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.2.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.2.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.2.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.2.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.2.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.2.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
"model.layers.2.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.20.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.20.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.20.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.20.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.20.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.20.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.20.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.20.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.20.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
"model.layers.20.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.21.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.21.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.21.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.21.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.21.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.21.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.21.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.21.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.21.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
"model.layers.21.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.22.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.22.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.22.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.22.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.22.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.22.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.22.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.22.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.22.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
"model.layers.22.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.23.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.23.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.23.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.23.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.23.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.23.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.23.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.23.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.23.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
"model.layers.23.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.24.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.24.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.24.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.24.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.24.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.24.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.24.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.24.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.24.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
"model.layers.24.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.25.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.25.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.25.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.25.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.25.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.25.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.25.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.25.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.25.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
"model.layers.25.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.26.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.26.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.26.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.26.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.26.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.26.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.26.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.26.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.26.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
"model.layers.26.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.27.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.27.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.27.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.27.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.27.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.27.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.27.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.27.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.27.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
"model.layers.27.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.28.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.28.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.28.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.28.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.28.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.28.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.28.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.28.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.28.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
"model.layers.28.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.29.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.29.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.29.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.29.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.29.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.29.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.29.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.29.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.29.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
"model.layers.29.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.3.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.3.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.3.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.3.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.3.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.3.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.3.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.3.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.3.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
"model.layers.3.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.30.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.30.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.30.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.30.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.30.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.30.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.30.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.30.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.30.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
"model.layers.30.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
"model.layers.31.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.31.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.31.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.31.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.31.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.31.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.31.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.31.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.31.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
"model.layers.31.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.32.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.32.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.32.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.32.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.32.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.32.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.32.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.32.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.32.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
"model.layers.32.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.33.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.33.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.33.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.33.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.33.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.33.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.33.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.33.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.33.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
"model.layers.33.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.34.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.34.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.34.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.34.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.34.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.34.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.34.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.34.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.34.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
"model.layers.34.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.35.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.35.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.35.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.35.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.35.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.35.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.35.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.35.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.35.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
"model.layers.35.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.36.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.36.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.36.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.36.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.36.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.36.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.36.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.36.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.36.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
"model.layers.36.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.37.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.37.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.37.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.37.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.37.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.37.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.37.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.37.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.37.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
"model.layers.37.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.38.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.38.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.38.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.38.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.38.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.38.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.38.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.38.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.38.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
"model.layers.38.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.39.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.39.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.39.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.39.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.39.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.39.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.39.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.39.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.39.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
"model.layers.39.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
"model.layers.4.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.4.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.4.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.4.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.4.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.4.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.4.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.4.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.4.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
"model.layers.4.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.5.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.5.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.5.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.5.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.5.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.5.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.5.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.5.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.5.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
"model.layers.5.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.6.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.6.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.6.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.6.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.6.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.6.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.6.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.6.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.6.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
"model.layers.6.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.7.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.7.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.7.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.7.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.7.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.7.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.7.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.7.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.7.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
"model.layers.7.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.8.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.8.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.8.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.8.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.8.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.8.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.8.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.8.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.8.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
"model.layers.8.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.9.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.9.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.9.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.9.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.9.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.9.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.9.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.9.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.layers.9.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
"model.layers.9.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
"model.norm.weight": "pytorch_model-00003-of-00003.bin"
}
}
You don't need a pytorch_model.bin.json for a GPTQ model. It doesn't do anything.
The error is caused by not having GPTQ settings chosen for the model. There was recently a bug in text-generation-webui that meant that GPTQ params weren't being saved properly. It's now been fixed, so update your text-gen-webui to the latest version if you're still having problems loading the model.
Ok, thank you for the update :)
Quick and dirty on a fresh install:
- Install ooba then update ooba (you need gcc installed, e.g. "apt install build-essential" or "Developer Tools" on Fedora)
- at the end use "Manually specify a Hugging Face model" to download "TheBloke/stable-vicuna-13B-GPTQ" and quit the installer, do not attempt to run it
- edit webui.py to use CMD_FLAGS = '--auto-devices --chat --wbits 4 --gpu-memory 22 --groupsize 128' instead of "CMD_FLAGS = '--chat --model-menu'" and start ooba
- set settings as outlined above or in README and reload model
You can then edit the webui.py back to what it was or what you prefer.
Possible pitfalls:
- you have not updated ooba
- you have no gcc installed
- you read this in a month or more from now
- you're lost in dependecy hell and noone can hear you scream
This was tested on a fresh install of ubuntu 22.04 wsl, gtx 3060 12GB, amd ryzen 5 3600
#edit: copy&paste error in flags
@TheBloke
This is more a potential error guide for new users that might come from a ooba install guide somewhere and have not downloaded or ran a model or if (in my case) you start troubleshooting from a clean slate. The reasoning behind this is that the ui won't start and you would never reach the settings inside the ui.
If stable-vicuna-13B-GPTQ is the first ever model downloaded the ui will not load. Normally you should be able to start the ui by selecting the "m" option, but that is currently not working ("you must select a model..."). Even if it would load the ui, it wouldnot immediately obvious that you would need to select this option after erroring out. That's all. It is however producing this exact error, so I wrote it here for posterity :)
Oh and the step 3 was stupidity on my side. I have since edited it to reflect the actual values.