Newbie question - My responses are all jibberish
Recently installed Oobabooga and ran with vicuna 13B ok. But running out of GPU memory - I have an RTX3080 (10GB)
Tried this model and when prompted, I just get text like this in response:
11har211zel1ò5hel181ide1111111 Mai111elen111 cura11 grasszelzel1zoròNa1òzel Har1dn1zelhar százHarharanguòharitalamò1haramilharhar har111òhar1FA1zelonom1harang curazelider cura1har1 cura1ò11 cura1har11zelharhar11har1harhar1harharid1òharhar1 curahar111 FA11harhar1 cura1harBackground1harideshar curailder11har curaœ191har1criv cura1pelzor11111 cura1 curaopo cura_+harhar1 cura1 cura1harächst7 EXISTS1harharidel1har1 curaNaNa1harzel11 curahar cura111 cura1harzelmd cura1har
What do I need to do in order to get sensible English responses?
Thank you!
This is caused by using too new of a GPTQ version, or too old. Follow these instructions for installing GPTQ https://github.com/oobabooga/text-generation-webui/wiki/LLaMA-model
Thank you so much for a kind and rapid response ❤️