Could you please create a q6 and q8 quant as well?
#2
by
hrishbhdalal
- opened
This will help the people who have 3x34GB setup. Big fan btw! Hope you guys finetune some models as well, apparently openhermese 2.5 beats llama3b instruct by quite some margin!
3x34? What devilry is this? 😂
Ahh.. pardon me. I mean 3x24GB (4090). I hope you guys train a killer model soon tho.. openhermese 3 maybe. The openhermese 2.5 is still better than Llama 3 in some benchmarks.