Can't lauch model using text-generation-webui
#1
by
azza696
- opened
Hey, thanks a lot for sharing this model.
I've been trying to test it using colab since it should fit their GPU (15360 GB VRAM)
But when trying to lauch the server using this command :
python server.py --model Thireus_Vicuna13B-v1.1-8bit-128g --model_type LLaMA --wbits 8 --groupsize 128 --share
The code returns without any message
bin /usr/local/lib/python3.9/dist-packages/bitsandbytes/libbitsandbytes_cuda118.so Loading Thireus_Vicuna13B-v1.1-8bit-128g... Found the following quantized model: models/Thireus_Vicuna13B-v1.1-8bit-128g/Vicuna13B-v1.1-8bit-128g.safetensors ^C
I've used both recommended and discouraged method but I have the same issue... Am i doing something wrong?
Here is the notebook btw : https://colab.research.google.com/drive/1BDOJ3qGypmhrYV3hlhEGNA8T3BxrY5WE?usp=sharing
azza696
changed discussion status to
closed
azza696
changed discussion status to
open