Better than Hermes.
#1
by
Delcos
- opened
This merge seems to preform better than the Hermes merge as well as the others tested. It also seems to lack the repetition issue from the Hermes merge, or at least it has less of an issue with it. Will post settings when they're finalized. Thanks for this one, it's impressively good.
no problem! and yeah it does seem to perform better. I think it could be because all the special tokens were both defined in the same way in both zephyr and neural chat, so they were probably trained with the same conversation template, while neural chat and open hermes seemed to use different special tokens.