avictus

/

oasst-sft-7-llama-30b-4bit

like 3

I am a bit new to using Webui, but I wanted to give this model a try to see how well it does. But every time I would try to load it I would get an error. It took me a while to realize that I had to update the webui.py

I updated the following line
'''
run_cmd("python server.py --chat --model-menu") # put your flags here!
'''
with
'''
run_cmd("python server.py --chat --model oasst-sft-7-llama-30b-4bit --wbits 4 --model_type llama")
'''

If anyone is curious the model requires >18GB of VRAM