Freezing Issue with gguf quant
#1
by
dillfrescott
- opened
It seems to be doing what every other openchat model that I've tried has done. It freezes occasionally in the middle of a generation.
It seems to be doing what every other openchat model that I've tried has done. It freezes occasionally in the middle of a generation.
same issue.
EOS token has to be set to "<|end_of_turn|>": 32000
This is not in the config.json, so gguf does not set it up by default.
We've updated config.json. Does this issue still happen?
We've updated config.json. Does this issue still happen?
It seems all right now. A great model! Looking forward to a more advanced version.
Okay closing this issue now. Re-open if it persists.
imone
changed discussion status to
closed