gorilla-llm

/

gorilla-openfunctions-v2

like 153

also has a problems with the chat template, I modified a little and ended like this (and worked like a charm)

{% if not add_generation_prompt is defined -%}
    {% set add_generation_prompt = false %}
{% endif %}
{%- set ns = namespace(found=false) -%}
{%- for message in messages -%}
    {%- if message['role'] == 'system' -%}
        {%- set ns.found = true -%}
    {%- endif -%}
{%- endfor -%}
{{ bos_token }}{%- if not ns.found -%}
{{ 'You are an AI programming assistant, utilizing the Deepseek Coder model, developed by Deepseek Company, and you only answer questions related to computer science. For politically sensitive questions, security and privacy issues, and other non-computer science questions, you will refuse to answer.' }}
{%- endif -%}
{%- for message in messages -%}
    {%- if message['role'] == 'system' -%}
{{- message['content'] }}
    {%- else -%}
        {%- if message['role'] == 'user' -%}
{{ '\n### Instruction:\n' + message['content'] + '\n' }}
        {%- else -%}
{{- '### Response:\n' + message['content'] + '\n\n' }}
        {%- endif -%}
    {%- endif -%}
{%- endfor -%}
{% if add_generation_prompt -%}
{{ '### Response:' }}
{% endif %}

Hey @prudant , sorry for the super late response. Thank you so much for building this 🔥 We reviewed your work and think it's very useful for the community. We welcome you to contribute to our main repo by raising a PR in https://github.com/ShishirPatil/gorilla/tree/main/openfunctions as an alternative inference script for more developers to use! Let us know if you have any questions before raising the PR! Thanks again.

And @prudant re-ing the chat template, thanks for looking into this! Could you provide a short example script demo-ing your use case with this chat template? We found that different chat templates are used in various packages, so we haven't updated the official chat_template and recommend developers use our official get_prompt(.) function instead as ground truth to avoid inconsistencies. Thanks in advance!

thanks, It would be an honor to be able to contribute with a grain of sand, if you explain a little about the folder structure of the repo or in which folder you would like this code to be, I will make the PR =)

best regards

for the chat template its a little bit more complex because I tested in rust inside the TGI (Hugging face text generation inference server), but my findings was the original jinja template was inyecting a lot nonsense spaces and line breaks, my version remove that with a better well formed jinja template, but its just that, i didnt touch the logic of the template, only its rendering format.

Thanks, @prudant ; the file structure goes like the following.
https://github.com/ShishirPatil/gorilla/tree/main/openfunctions contains the most up-to-date gorilla-openfunctions-v2 inference code, please include your openai.py (please rename this to have consistent naming with other files) in this folder. If you have utils files, please store them at https://github.com/ShishirPatil/gorilla/tree/main/openfunctions/utils.
Then, open a new section in https://github.com/ShishirPatil/gorilla/blob/main/openfunctions/README.md to include instructions for running this script, an example, and the next steps of your work if this is an ongoing effort.

Thanks again!

Hello, I have been away for a while due to health problems with hospitalization, at this moment I read some emails and I remembered you, since I am away feel free to add my code to your repository, I am a little complicated with doing it for a while.

The only thing I ask is to mention the author =)

REGARDS!