Update tool use template to new format
#46
by
Rocketknight1
HF staff
- opened
Do not merge this yet! This is a currently experimental PR to test a new template format for tool use, to go with the new JSON schema standard for passing tools. It won't work correctly until that PR is also merged.
To try it out, you should install from the PR branch with pip install --upgrade git+https://github.com/huggingface/transformers.git@new_chat_template_args
Then, load the tokenizer and template from this PR branch:
from transformers import AutoTokenizer
from typing import List, Dict
tokenizer = AutoTokenizer.from_pretrained("CohereForAI/c4ai-command-r-plus", revision="pr/46")
def internet_search(query: str):
"""Returns a list of relevant document snippets for a textual query retrieved from the internet
Args:
query: Query to search the internet with
"""
pass
def directly_answer():
"""Calls a standard (un-augmented) AI chatbot to generate a response given the conversation history
"""
pass
conversation = [
{"role": "user", "content": "Whats the biggest penguin in the world?"}
]
tools = [internet_search, directly_answer]
new_tool_use_prompt = tokenizer.apply_chat_template(
conversation,
tools=tools,
chat_template="tool_use",
tokenize=False,
add_generation_prompt=True,
)
print(new_tool_use_prompt)