I have converted the thing in GPTQ-v2

Group size 128 for one

Act order for another

and neither for a third

Let's see how many I upload.

Also converted to GGML in the GGML folder. Works on CPU.

Downloads last month
168
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support