Wow!
#1
by
Nexesenex
- opened
Serious upload of my most awaited 16bits quants!
I'm downloading the bf16, and cross fingers for this version to work as intended!
Thank you very much, and kudos for the model card!
Thank you!
Hope it goes well, I can only spot-check the lower quants on my end, unfortunately.
I usually remake a custom quant from a fp16 (LLama3) or Q8_0 (other models) to get the best of my computer (36GB VRAM).
The v1 is amazing (still my main model), the v3 had troubles, the 3.5 had initially the wrong tokenizet (fixed yesterday failspy yesterday, it had Smaug tokenizer instead of L3), let's hope this it it!