EOS token
#1
by
jpohhhh
- opened
The EOS token in the GGUF file is <|endoftext|>
The model never seems to generate it, so it generates "infinite" dialogue instead of turn-based.
I'm guessing the stop phrase is something like "\n###"
Yes, the chat format is:
### User:
[text]
### Assistant:
[more text]
### User:
[turn 2 text]
### Assistant:
[etc.]
\n### would work well.
You can also prompt it for rationale+answer generation:
### User:
[text]
### Rationale:
[more text]
### Answer:
[direct answer based on the reasoning above]<|endoftext|>