Run the model with oobabooga/text-generation-webui - missing config.json, etc.
N00b here!
When I try to use .safetensors model file with oobabooga, it says I'm missing config.json, and probably all the other files like tokenizer.model that usually come along.
Can I use these from the older vicuna-13b-4bit-128g model that anon8231489123 released before, or do I need new ones specifically for this model, and where do I get them?
Thanks.
Yes I had the same issue and it worked putting in those default files from the original.
https://huggingface.co/anon8231489123/vicuna-13b-GPTQ-4bit-128g/tree/main
I put all these in the same folder with it.
config.json
generation_config.json
pytorch_model.bin.index.json
special_tokens_map.json
tokenizer.model
tokenizer_config.json
I seem to run into the following issue when loading the model using the configs from the above repo
RuntimeError: Error(s) in loading state_dict for LlamaForCausalLM:
size mismatch for model.embed_tokens.weight: copying a param with shape torch.Size([32001, 5120]) from checkpoint, the shape in current model is torch.Size([32000, 5120]).
size mismatch for lm_head.weight: copying a param with shape torch.Size([32001, 5120]) from checkpoint, the shape in current model is torch.Size([32000, 5120]).
OK, I think I sorted that by just editing config.json and changing 32000 to 32001
Thanks, it worked using files from anon9231489123, but now I'm getting some strange initial responses so I'll open another issue about that.
For the new model, you need to change from 32001 to 32000 again to make it work.