chat template issue
When running the given build in the llama.cpp environment, no errors appeared on the llama-server's own page, but the following error occurs in other frontends that support OpenAI compatibility.
W srv operator (): got exception: {"error":{"code":500,"message":"\n------------\nWhile executing CallExpression at line 95, column 32 in source:\n...first %}↵ {{- raise_exception('System message must be at the beginnin...\n ^\nError: Jinja Exception: System message must be at the beginning.","type":"server_error"}}
This clause stipulates that system messages must appear first, but this causes conflicts and creates problems. So, I tried to fix it myself to prevent these errors from occurring while preserving the existing template as much as possible.
{%- set truncate_history_thinking = truncate_history_thinking if truncate_history_thinking is defined else false %}
{%- macro render_content(content, is_system_content=false) %}
{%- if content is string %}
{{- content }}
{%- elif content is iterable and content is not mapping %}
{%- for item in content %}
{%- if 'text' in item %}
{{- item.text }}
{%- else %}
{{- raise_exception('raon-1-mini supports text content only.') }}
{%- endif %}
{%- endfor %}
{%- elif content is none or content is undefined %}
{{- '' }}
{%- else %}
{{- raise_exception('Unexpected content type.') }}
{%- endif %}
{%- endmacro %}
{%- macro render_assistant_body(message, content, reasoning_content, preserve_reasoning) %}
{%- if preserve_reasoning and reasoning_content|trim %}
{{- '<think>\n' + reasoning_content|trim + '\n</think>\n\n' }}
{%- else %}
{{- '<think></think>\n\n' }}
{%- endif %}
{{- content }}
{%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
{%- for tool_call in message.tool_calls %}
{%- if tool_call.function is defined %}
{%- set tool_call = tool_call.function %}
{%- endif %}
{%- if loop.first %}
{%- if content|trim %}
{{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
{%- else %}
{{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
{%- endif %}
{%- else %}
{{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
{%- endif %}
{%- if tool_call.arguments is defined %}
{%- for args_name, args_value in tool_call.arguments|items %}
{{- '<parameter=' + args_name + '>\n' }}
{%- set args_value = args_value | string if args_value is string else args_value | tojson | safe %}
{{- args_value }}
{{- '\n</parameter>\n' }}
{%- endfor %}
{%- endif %}
{{- '</function>\n</tool_call>' }}
{%- endfor %}
{%- endif %}
{{- '<|im_end|>' }}
{%- endmacro %}
{%- if not messages %}
{{- raise_exception('No messages provided.') }}
{%- endif %}
{%- set sys_presence = namespace(has=false) %}
{%- for message in messages %}
{%- if message.role == 'system' %}
{%- set sys_presence.has = true %}
{%- endif %}
{%- endfor %}
{%- if tools and tools is iterable and tools is not mapping %}
{{- '<|im_start|>system\n' }}
{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}
{%- for tool in tools %}
{{- '\n' }}
{{- tool | tojson }}
{%- endfor %}
{{- '\n</tools>' }}
{{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
{%- set sys_parts = namespace(has_previous=true) %}
{%- for message in messages %}
{%- if message.role == 'system' %}
{%- set content = render_content(message.content, true)|trim %}
{%- if content %}
{%- if sys_parts.has_previous %}
{{- '\n\n' }}
{%- endif %}
{{- content }}
{%- set sys_parts.has_previous = true %}
{%- endif %}
{%- endif %}
{%- endfor %}
{{- '<|im_end|>\n' }}
{%- elif sys_presence.has %}
{{- '<|im_start|>system\n' }}
{%- set sys_parts = namespace(has_previous=false) %}
{%- for message in messages %}
{%- if message.role == 'system' %}
{%- set content = render_content(message.content, true)|trim %}
{%- if content %}
{%- if sys_parts.has_previous %}
{{- '\n\n' }}
{%- endif %}
{{- content }}
{%- set sys_parts.has_previous = true %}
{%- endif %}
{%- endif %}
{%- endfor %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for message in messages[::-1] %}
{%- set index = (messages|length - 1) - loop.index0 %}
{%- if ns.multi_step_tool and message.role == 'user' %}
{%- set content = render_content(message.content)|trim %}
{%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
{%- set ns.multi_step_tool = false %}
{%- set ns.last_query_index = index %}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- if ns.multi_step_tool %}
{{- raise_exception('No user query found in messages.') }}
{%- endif %}
{%- for message in messages %}
{%- set content = render_content(message.content)|trim %}
{%- if message.role == 'system' %}
{{- '' }}
{%- elif message.role == 'user' %}
{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}
{%- elif message.role == 'assistant' %}
{%- set reasoning_content = '' %}
{%- if message.reasoning_content is string %}
{%- set reasoning_content = message.reasoning_content %}
{%- elif '</think>' in content %}
{%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
{%- set content = content.split('</think>')[-1].lstrip('\n') %}
{%- endif %}
{{- '<|im_start|>assistant\n' }}
{%- set preserve_reasoning = (not truncate_history_thinking) or (loop.index0 > ns.last_query_index) %}
{%- if message.loss is defined and message.loss is false %}
{{- render_assistant_body(message, content, reasoning_content, preserve_reasoning) }}
{%- else %}
{%- generation %}
{{- render_assistant_body(message, content, reasoning_content, preserve_reasoning) }}
{%- endgeneration %}
{%- endif %}
{{- '\n' }}
{%- elif message.role == 'tool' %}
{%- if loop.previtem and loop.previtem.role != 'tool' %}
{{- '<|im_start|>user' }}
{%- endif %}
{{- '\n<tool_response>\n' + content + '\n</tool_response>' }}
{%- if loop.last or loop.nextitem.role != 'tool' %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- else %}
{{- raise_exception('Unexpected message role.') }}
{%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|im_start|>assistant\n' }}
{%- if enable_thinking is defined and enable_thinking is false %}
{{- '<think></think>\n\n' }}
{%- else %}
{{- '<think>\n' }}
{%- endif %}
{%- endif %}
I hope this amendment will be helpful to others as well.
Hi bamec66557, thanks for catching this and taking the time to share a fix!
We’ve updated the template to allow system messages in the middle of a conversation.
I went with a slightly different approach: each system message stays where it was sent, so the original message order is preserved.
Let me know if you still run into any issues. Thanks again!
Thank you for considering my minor suggestion. I’m currently using it without any issues, and the problem I mentioned has been resolved. As a follow-up suggestion, I tested the system in different ways using the newly provided templates, but unfortunately, I discovered that another issue arises. It’s not a major error, but front-end tools like TextGen, Tavern, and Silly Tavern have features that allow the system to impersonate the user’s responses to continue the conversation. With the current templates, the following error occurs:
W srv operator (): got exception: {"error":{"code":500,"message":"\n------------\nWhile executing CallExpression at line 89, column 24 in source:\n...lti_step_tool %}↵ {{- raise_exception('No user query found in messages.') }}↵...\n ^\nError: Jinja Exception: No user query found in messages.","type":"server_error"}}
The code in lines 77–90, which is
{%- if ns.multi_step_tool %}
{{- raise_exception('No user query found in messages.') }}
{%- endif %}
It appears that if there are no user questions or messages, this section returns the value as-is, causing an error. I made a simple modification to this section as shown below and am currently using it.
{%- if ns.multi_step_tool %}
{%- set ns.last_query_index = -1 %}
{%- endif %}
Please review this section. Additionally, I am impressed by the current model's features and hope to see further improvements to its already outstanding performance. I hope this suggestion is helpful.
Thanks for the additional feedback!
I reviewed your suggestion and agree that there’s no need to raise an exception when a conversation has no user message.
I’ve removed that check and updated the chat template, so it should resolve the error you reported.
Please give it a try when you have a chance, and let me know if you run into anything else.
Thanks again, and please stay tuned for our upcoming models!