This PR aims to align the tokenizer_config to allow the latest changes in HF tokenizer to be propagated here.
This is rough first version of the template based on my understanding of the way your tokenizer works ( append available tools right before the last user message)

What do you think ?
cc: @Rocketknight1 , @pstock , @sophiamyang , @osanseviero

Example use :

1) With tools :

from transformers import AutoTokenizer
model_id = "mistralai/Mixtral-8x22B-Instruct-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
conversation=[
    {"role": "user", "content": "What's the weather like in Paris?"},
    {
        "role": "tool_calls",
        "content": [
            {
                "name": "get_current_weather",
                "arguments": {"location": "Paris, France", "format": "celsius"},
                "id": "VvvODy9mT"
            }
        ]
    },
    {
        "role": "tools_results",
        "content": {"call_id": "VvvODy9mT", "content": 22}
    },
    {"role": "assistant", "content": "The current temperature in Paris, France is 22 degrees Celsius."},
    {"role": "user", "content": "What about San Francisco?"}
]


tools = [{"type": "function", "function": {"name":"get_current_weather", "description": "Get_the_current_weather", "parameters": {"type": "object", "properties": {"location": {"type": "string", "description": "The city and state, e.g. San Francisco, CA"}, "format": {"type": "string", "enum": ["celsius", "fahrenheit"], "description": "The temperature unit to use. Infer this from the users location."}},"required":["location","format"]}}}]

# render the tool use prompt as a string:
tool_use_prompt = tokenizer.apply_chat_template(
            conversation,
            chat_template="tool_use",
            tools=tools,
            tokenize=False,
            add_generation_prompt=True,

)
print(tool_use_prompt)

Which renders :

<s>[INST]What's the weather like in Paris?[/INST][TOOL_CALLS][{'name': 'get_current_weather', 'arguments': {'location': 'Paris, France', 'format': 'celsius'}, 'id': 'VvvODy9mT'}]</s>[/TOOL_CALLS] The current temperature in Paris, France is 22 degrees Celsius. </s>[AVAILABLE_TOOLS]{'type': 'function', 'function': {'name': 'get_current_weather', 'description': 'Get_the_current_weather', 'parameters': {'type': 'object', 'properties': {'location': {'type': 'string', 'description': 'The city and state, e.g. San Francisco, CA'}, 'format': {'type': 'string', 'enum': ['celsius', 'fahrenheit'], 'description': 'The temperature unit to use. Infer this from the users location.'}}, 'required': ['location', 'format']}}}[/AVAILABLE_TOOLS][INST]What about San Francisco?[/INST]

2) Without tools:

from transformers import AutoTokenizer

# Initialize model ID and tokenizer
model_id = "mistralai/Mixtral-8x22B-Instruct-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Define a conversation example
conversation = [
    {"role": "user", "content": "What's the weather like in Paris?"},
    {"role": "assistant", "content": "The current temperature in Paris, France is 22 degrees Celsius."},
    {"role": "user", "content": "What about San Francisco?"}
]

# Generate the tool use prompt using the tokenizer
tool_use_prompt = tokenizer.apply_chat_template(
    conversation,
    tokenize=False,
    add_generation_prompt=True
)

# Print the generated prompt
print(tool_use_prompt)

Which renders :

<s> [INST] What's the weather like in Paris? [/INST] The current temperature in Paris, France is 22 degrees Celsius. </s> [INST] What about San Francisco? [/INST]

Thanks so much for the PR! The format with tools is a little off. We don't have [/TOOL_CALLS].

Thanks @sophiamyang , it seems I "halucinated" that token.

Should be better now.

Thanks! merged.

@Jofthomas I'm a little confused why it didn't seem to work after I merge. Does it need time to make changes when I run the code?

Doesn't this code work for you ? @sophiamyang

It does for me ( i just tested it ) :

Tools use :

from transformers import AutoTokenizer
model_id = "mistralai/Mixtral-8x22B-Instruct-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
conversation=[
    {"role": "user", "content": "What's the weather like in Paris?"},
    {
        "role": "tool_calls",
        "content": [
            {
                "name": "get_current_weather",
                "arguments": {"location": "Paris, France", "format": "celsius"},
                "id": "VvvODy9mT"
            }
        ]
    },
    {
        "role": "tool_results",
        "content": {"call_id": "VvvODy9mT", "content": 22}
    },
    {"role": "assistant", "content": "The current temperature in Paris, France is 22 degrees Celsius."},
    {"role": "user", "content": "What about San Francisco?"}
]


tools = [{"type": "function", "function": {"name":"get_current_weather", "description": "Get▁the▁current▁weather", "parameters": {"type": "object", "properties": {"location": {"type": "string", "description": "The city and state, e.g. San Francisco, CA"}, "format": {"type": "string", "enum": ["celsius", "fahrenheit"], "description": "The temperature unit to use. Infer this from the users location."}},"required":["location","format"]}}}]

# render the tool use prompt as a string:
tool_use_prompt = tokenizer.apply_chat_template(
            conversation,
            chat_template="tool_use",
            tools=tools,
            tokenize=False,
            add_generation_prompt=True,

)
print(tool_use_prompt)

No tools :

from transformers import AutoTokenizer

# Initialize model ID and tokenizer
model_id = "mistralai/Mixtral-8x22B-Instruct-v0.1"
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Define a conversation example
conversation = [
    {"role": "user", "content": "What's the weather like in Paris?"},
    {"role": "assistant", "content": "The current temperature in Paris, France is 22 degrees Celsius."},
    {"role": "user", "content": "What about San Francisco?"}
]

# Generate the tool use prompt using the tokenizer
tool_use_prompt = tokenizer.apply_chat_template(
    conversation,
    tokenize=False,
    add_generation_prompt=True
)

# Print the generated prompt
print(tool_use_prompt)

Copied and pasted your code. Just ran it again. I'm not sure why it doesn't work for me @Jofthomas

You should upgrade transformers as it is part of the latest change :)

pip install transformers==4.40.0

This works for version >= 4.39.0

Thanks @Jofthomas ! That works great. One more thing: it should be a list after [AVAILABLE_TOOLS]

ohh I see, let me update the template in a seperate PR, and I'll also update the Readme so that users can have an example of how to use it and do not fall into the same version problem.

Does it sounds good ?

great thanks!

have folks had success getting back a valid function calling response from the model with this template?