I made a mild oopsie.
When uploading the initial files i forgot to update the config files initially, and it appears these quants managed to catch the outdated configs. I can pr the changes to the configs if that would help, the quants shouldn't need to be remade. The 0.6 gamma configs are correct, but the model is trained on both ChatML and l3 instruct.
For exl2 it should be enough to just update the config files, for GGUF I'll probably want to remake it from scratch
For exl2 it should be enough to just update the config files, for GGUF I'll probably want to remake it from scratch
The configs from 0.6 should work fine for 0.5 as well.
Wait what's the difference between 0.6 and 0.5? They both use the L3 prompt format instead of ChatML
Wait what's the difference between 0.6 and 0.5? They both use the L3 prompt format instead of ChatML
0.6 was trained on ChatML, but seems to work better in l3 instruct still. So, i left the configs to reflect that. (however 0.6 suffers from some other problems), part of the reason i moved it over to the archive org. 0.5 Seems more stable and was trained on l3-instruct format.