openchat

/

openchat_3.5

like 1.09k

EOS should be 32000, otherwise generation doesn't terminate on the first <|endofturn|> token, making it seem like generation has frozen (in fact it's generating endless <|endofturn|> tokens until it reaches the max token limit)

Reference for the issue: https://huggingface.co/TheBloke/openchat_3.5-GGUF/discussions/1#6544063ae5c235427a3abde9