lightblue

/

Karasu-Mixtral-8x22B-v0.1-gguf

like 14

So this is a port of this model: lightblue/Karasu-Mixtral-8x22B-v0.1 and the chat template is simply the mistral template (as outlined here).

The Jinja template is as follows:

{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token}}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}

I don't have any experience with Ollama, but looking at their docs, the template should be something like:

[INST] {{ .System }} {{ .Prompt }} [/INST]

Refer to original Mistral Ollama template