Hello,
Like the title says, I'm wondering if this is running in 16bit or 4bit.
Thanks.
I cannot verify what I am saying but since it is said that it was trained using QLora, it is most certainly a 4bit quantized model.
· Sign up or log in to comment
