EXL2 quants of Smaug-72B-v0.1.
Couldn't find a description of the prompt format, but it looks like OpenChat?
2.50 bits per weight (may be a little broken?)
3.00 bits per weight
4.00 bits per weight
5.00 bits per weight
EXL2 quants of Smaug-72B-v0.1.
Couldn't find a description of the prompt format, but it looks like OpenChat?
2.50 bits per weight (may be a little broken?)
3.00 bits per weight
4.00 bits per weight
5.00 bits per weight