Is it possible to load this in Kobold 4bit?
#5
by
ShatteredBlue
- opened
I followed this in the other repo : https://huggingface.co/TheBloke/wizardLM-7B-GPTQ/discussions/6
And it worked wonderfully.
According to KoboldAI I would've been fine renaming that file to 4bit-128g.safetensors but doesn't seem to work in this case. I'm wondering if a .pt file similar to the one mentioned above might be possible?
I kinda threw in the towel with ooba since I don't know what I'm doing and running on limited bandwidth. It has something to do with Gardio which puzzles me because last time I used it, it worked fine but this was a while back.