This is the other model that had the same problem to my knowledge.
#1
by
Nitral-AI
- opened
This is the other model that had the same problem to my knowledge, as lelantos 7b dpo discussed earlier.
I have not been able to recreate the never ending generation issue on any Noromaid but it is consistent on the Lelantos-Maids.
I have tried various changes on the config files and remade the quant but still the same issue.
I will private the Lelantos-Maids since i cant find a solution right now. Please tell me if you figure something out.
Redownloading the noromaid 0.4 4bpw to attempt and recreate again, weird enough token generation seems to be handling better on lelantos-maid in alpaca. which is strange... Thank you for all the effort though, i do appreciate it alot.
Nitral-AI
changed discussion status to
closed