Hi, what are the setting to try to load this into oogabooga webui? Thanks
#1
by
Humeee33
- opened
I know it's not 128 or 4 bit. Thanks
I was able to get this one to work. https://huggingface.co/gozfarb/llygmalion-13B-4bit-128g-cuda
The 4-bit version. I just had to change the model type from none to llama.