Hugging Face's logo

bartowski
/
Qwen_QwQ-32B-GGUF

Text Generation
GGUF
English
chat
Inference Endpoints
imatrix
conversational
Model card Files Community
8

Error in LMStudio:Error rendering prompt with jinja template: Error: Parser Error: Expected closing statement token. OpenSquareBracket !== CloseStatement.

#5
by LightDust - opened
Discussion
LightDust

I got this error while I was running Qwen_QwQ-32B-Q4_K_M.gguf using LMstudio
Error rendering prompt with jinja template: Error: Parser Error: Expected closing statement token. OpenSquareBracket !== CloseStatement.
I cant use it, even like "Hello".

svdvdsv
•
edited 3 days ago

Go to my Models
Click on the gear next to the model
Prompts on the right
and replace with

{%- if tools %} {{- '<|im_start|>system\n' }} {%- if messages[0]['role'] == 'system' %} {{- messages[0]['content'] }} {%- else %} {{- '' }} {%- endif %} {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within XML tags:\n" }} {%- for tool in tools %} {{- "\n" }} {{- tool | tojson }} {%- endfor %} {{- "\n\n\nFor each function call, return a json object with function name and arguments within XML tags:\n\n{"name": , "arguments": }\n<|im_end|>\n" }} {%- else %} {%- if messages[0]['role'] == 'system' %} {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }} {%- endif %} {%- endif %} {%- for message in messages %} {%- if (message.role == "user") or (message.role == "system" and not loop.first) %} {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }} {%- elif message.role == "assistant" and not message.tool_calls %} {%- set content = (message.content.split('')|last).lstrip('\n') %} {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }} {%- elif message.role == "assistant" %} {%- set content = (message.content.split('')|last).lstrip('\n') %} {{- '<|im_start|>' + message.role }} {%- if message.content %} {{- '\n' + content }} {%- endif %} {%- for tool_call in message.tool_calls %} {%- if tool_call.function is defined %} {%- set tool_call = tool_call.function %} {%- endif %} {{- '\n\n{"name": "' }} {{- tool_call.name }} {{- '", "arguments": ' }} {{- tool_call.arguments | tojson }} {{- '}\n' }} {%- endfor %} {{- '<|im_end|>\n' }} {%- elif message.role == "tool" %} {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %} {{- '<|im_start|>user' }} {%- endif %} {{- '\n\n' }} {{- message.content }} {{- '\n' }} {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %} {{- '<|im_end|>\n' }} {%- endif %} {%- endif %} {%- endfor %} {%- if add_generation_prompt %} {{- '<|im_start|>assistant\n' }} {%- endif %}

ckoguchi

@svdvdsv that didn't quite work for me, but it helped me get in the right direction. Here's a Jinja template that worked for me. Seems like it might have just been caused by the unescaped double quotes inside the JSON snippet here: "\n\n\nFor each function call, return a json object with function name and arguments within XML tags:\n\n{"name": , "arguments": }\n<|im_end|>\n"

Copy Paste:

{%- if tools %}
  {{- '<|im_start|>system\n' }}
  {%- if messages[0]['role'] == 'system' %}
    {{- messages[0]['content'] }}
  {%- else %}
    {{- '' }}
  {%- endif %}
  {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within XML tags:\n" }}
  {%- for tool in tools %}
    {{- "\n" }}
    {{- tool | tojson }}
  {%- endfor %}
  {{- "\n\n\nFor each function call, return a json object with function name and arguments within XML tags:\n\n{\"name\": , \"arguments\": }\n<|im_end|>\n" }}
{%- else %}
  {%- if messages[0]['role'] == 'system' %}
    {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
  {%- endif %}
{%- endif %}
{%- for message in messages %}
  {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
    {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
  {%- elif message.role == "assistant" and not message.tool_calls %}
    {%- set content = (message.content.split('')|last).lstrip('\n') %}
    {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
  {%- elif message.role == "assistant" %}
    {%- set content = (message.content.split('')|last).lstrip('\n') %}
    {{- '<|im_start|>' + message.role }}
    {%- if message.content %}
      {{- '\n' + content }}
    {%- endif %}
    {%- for tool_call in message.tool_calls %}
      {%- if tool_call.function is defined %}
        {%- set tool_call = tool_call.function %}
      {%- endif %}
      {{- '\n\n{"name": "' }}
      {{- tool_call.name }}
      {{- '", "arguments": ' }}
      {{- tool_call.arguments | tojson }}
      {{- '}\n' }}
    {%- endfor %}
    {{- '<|im_end|>\n' }}
  {%- elif message.role == "tool" %}
    {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
      {{- '<|im_start|>user' }}
    {%- endif %}
    {{- '\n\n' }}
    {{- message.content }}
    {{- '\n' }}
    {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
      {{- '<|im_end|>\n' }}
    {%- endif %}
  {%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
  {{- '<|im_start|>assistant\n' }}
{%- endif %}
ckoguchi

Actually, I see this is probably a duplicate issue and they have guidance there for how to fix it. Tried it out myself and it works!
https://huggingface.co/bartowski/Qwen_QwQ-32B-GGUF/discussions/4#67c8e8317d3aea697215f8ce

LightDust
•
edited 2 days ago

thx

LightDust
This comment has been hidden
LightDust
This comment has been hidden
LightDust changed discussion status to closed
LightDust changed discussion status to open
LightDust

@svdvdsv that didn't quite work for me, but it helped me get in the right direction. Here's a Jinja template that worked for me. Seems like it might have just been caused by the unescaped double quotes inside the JSON snippet here: "\n\n\nFor each function call, return a json object with function name and arguments within XML tags:\n\n{"name": , "arguments": }\n<|im_end|>\n"

Copy Paste:

{%- if tools %}
  {{- '<|im_start|>system\n' }}
  {%- if messages[0]['role'] == 'system' %}
    {{- messages[0]['content'] }}
  {%- else %}
    {{- '' }}
  {%- endif %}
  {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within XML tags:\n" }}
  {%- for tool in tools %}
    {{- "\n" }}
    {{- tool | tojson }}
  {%- endfor %}
  {{- "\n\n\nFor each function call, return a json object with function name and arguments within XML tags:\n\n{\"name\": , \"arguments\": }\n<|im_end|>\n" }}
{%- else %}
  {%- if messages[0]['role'] == 'system' %}
    {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
  {%- endif %}
{%- endif %}
{%- for message in messages %}
  {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
    {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
  {%- elif message.role == "assistant" and not message.tool_calls %}
    {%- set content = (message.content.split('')|last).lstrip('\n') %}
    {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
  {%- elif message.role == "assistant" %}
    {%- set content = (message.content.split('')|last).lstrip('\n') %}
    {{- '<|im_start|>' + message.role }}
    {%- if message.content %}
      {{- '\n' + content }}
    {%- endif %}
    {%- for tool_call in message.tool_calls %}
      {%- if tool_call.function is defined %}
        {%- set tool_call = tool_call.function %}
      {%- endif %}
      {{- '\n\n{"name": "' }}
      {{- tool_call.name }}
      {{- '", "arguments": ' }}
      {{- tool_call.arguments | tojson }}
      {{- '}\n' }}
    {%- endfor %}
    {{- '<|im_end|>\n' }}
  {%- elif message.role == "tool" %}
    {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
      {{- '<|im_start|>user' }}
    {%- endif %}
    {{- '\n\n' }}
    {{- message.content }}
    {{- '\n' }}
    {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
      {{- '<|im_end|>\n' }}
    {%- endif %}
  {%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
  {{- '<|im_start|>assistant\n' }}
{%- endif %}

It's effective! thank you

LightDust changed discussion status to closed

Sign up or log in to comment