unsloth

/

llama-3-8b-bnb-4bit

like 115

I noticed that the tokenizer does not have a chat_template defined. So I created one based on the details available here

tokenizer.chat_template = (
    "{{bos_token}}"
    "{% for message in messages %}"
        "<|start_header_id|>{{message['role']}}<|end_header_id|>\n\n{{message['content']}}<|eot_id|>"
    "{% endfor %}"
    "{% if add_generation_prompt%}<|start_header_id|>assistant<|end_header_id|>\n\n"
        "{% else %}{{eos_token}}"
    "{% endif %}"
)

chat = [
    {"role": "system", "content": "You are a very useful assistant"},
    {"role": "assistant", "content": "I'm doing great. How can I help you today?"},
    {"role": "user", "content": "I'd like to show off how chat templating works!"},
]

print(tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True))

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a very useful assistant<|eot_id|><|start_header_id|>assistant<|end_header_id|>

I'm doing great. How can I help you today?<|eot_id|><|start_header_id|>user<|end_header_id|>

I'd like to show off how chat templating works!<|eot_id|><|start_header_id|>assistant<|end_header_id|>