Spaces:
Running
on
Zero
Custom Format for Mistral
When i saw the llama-cpp-agent, the mistral prompt makers is looks like combining v1 and tekken chat template.
Based on Mistral Cookbook and inflatebot's SillyTavern template
i generate this code with LLM
from llama_cpp_agent.messages_formatter import MessagesFormatter, PromptMarkers, Roles
mistral_v1_markers = {
Roles.system: PromptMarkers(""" [INST] """, """ [/INST] Understood.</s>"""),
Roles.user: PromptMarkers(""" [INST] """, """ [/INST]"""),
Roles.assistant: PromptMarkers(" ", "</s>"),
Roles.tool: PromptMarkers("", ""),
}
mistral_v2_markers = {
Roles.system: PromptMarkers("""[INST] """, """[/INST] Understood.</s>"""),
Roles.user: PromptMarkers("""[INST] """, """[/INST]"""),
Roles.assistant: PromptMarkers(" ", "</s>"),
Roles.tool: PromptMarkers("", ""),
}
mistral_v3_tekken_markers = {
Roles.system: PromptMarkers("""[INST]""", """[/INST]Understood.</s>"""),
Roles.user: PromptMarkers("""[INST]""", """[/INST]"""),
Roles.assistant: PromptMarkers("", "</s>"),
Roles.tool: PromptMarkers("", ""),
}
mistral_v1_formatter = MessagesFormatter(
pre_prompt="",
prompt_markers=mistral_v1_markers,
include_sys_prompt_in_first_user_message=False,
default_stop_sequences=["</s>"]
)
mistral_v2_formatter = MessagesFormatter(
pre_prompt="",
prompt_markers=mistral_v2_markers,
include_sys_prompt_in_first_user_message=False,
default_stop_sequences=["</s>"]
)
mistral_v3_tekken_formatter = MessagesFormatter(
pre_prompt="",
prompt_markers=mistral_v3_tekken_markers,
include_sys_prompt_in_first_user_message=False,
default_stop_sequences=["</s>"]
)
i haven't tested it but maybe it's useful
Thanks for the contribution! I made a bit of a mistake at first, but your part of the code seems to work just fine as is. This change also makes it easier to add templates in the future.
I read the code on github when I bugged it, and it seems that the preset templates are not that complex to define. We can customize it without worry.
https://github.com/Maximilian-Winter/llama-cpp-agent/blob/master/src/llama_cpp_agent/messages_formatter.py
Glad to hear that.
Well, I think it's also great if it can be moved upstream
I'll have to get a github account first!
Well, the acquisition itself may be easy, but I can't take the time to get introduced to github.๐ฅฒ
The modification of the github code itself is easy, the rest of the work is just to decide on the official name. The capitalized one.
I got a github account and PR'd it.
https://github.com/Maximilian-Winter/llama-cpp-agent/pull/82
Nice work