teknium

/

OpenHermes-2.5-Mistral-7B

like 816

Hello!

I came across this model and I must say, I'm very impressed with the results.
While examining the tokenizer_config.json file, I noticed that it specifies < s > as the BOS-token and add_bos_token:true.

Although the model card mentions using the ChatML format, this raised some questions that I hope to receive assistance with.
How should < s > be specified in the prompt/prompts when chatting with the model?

Example 1 - Not at all.
"<|im_start|>User
Question 1?<|im_end|>
<|im_start|>Assistant
Answer 1.<|im_end|>
<|im_start|>User
Question 2 based on question 1?<|im_end|>
<|im_start|>Assistant"

Example 2 - Before each message/prompt.
"< s > <|im_start|>User
I'm human<|im_end|>
< s > <|im_start|>Assistant
I'm Assistant<|im_end|>
< s > |im_start|>User
I'm human<|im_end|>
< s >|im_start|>Assistant
"
Example 3 - At the start of the sequence.
"< s > <|im_start|>User
Question 1?<|im_end|>
<|im_start|>Assistant
Answer 1<|im_end|>
<|im_start|>User
Question 2 based on question 1?<|im_end|
<|im_start|>Assistant"

Thanks.

edit. I can't write the proper BOS-token for some reason without formatting the whole text on here. I've added spaces to make it appear in the text.