What are the different files for?

#9
by Arya123456 - opened

I am a beginner at this and was wondering which of these files is to be used in the Oobabooga WebUI? Or do I need all of them? Thanks for your help :)

gpt4-x-alpasta-30b-128g-4bit.safetensors

gpt4-x-alpasta-30b-4bit.safetensors

gpt4-x-alpasta-30b-ggml-q4_1.bin

gpt4-x-alpasta-30b-ggml-q5_0.bin

gpt4-x-alpasta-30b-ggml-q5_1.bin

I think you use the highest one your card/VRAM can support. 5_1 would be the best for the BINS, but if you can use the safetensor without the 128g in it that would be ideal, as it won't exceed your VRAM is how I understand it.

Thank you Clevnumb!

Sign up or log in to comment