Chat
#2
by
jiyintor
- opened
Did you fine-tune the model? Could you share the TEMPLATE
compatible with Ollama?
So this is a port of this model: lightblue/Karasu-Mixtral-8x22B-v0.1 and the chat template is simply the mistral template (as outlined here).
The Jinja template is as follows:
{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token}}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}
I don't have any experience with Ollama, but looking at their docs, the template should be something like:
[INST] {{ .System }} {{ .Prompt }} [/INST]