CohereForAI

/

c4ai-command-r-plus

like 1.44k

Do not merge this yet! This is a currently experimental PR to test a new template format for tool use, to go with the new JSON schema standard for passing tools. It won't work correctly until that PR is also merged.

To try it out, you should install from the PR branch with pip install --upgrade git+https://github.com/huggingface/transformers.git@new_chat_template_args

Then, load the tokenizer and template from this PR branch:

from transformers import AutoTokenizer
from typing import List, Dict

tokenizer = AutoTokenizer.from_pretrained("CohereForAI/c4ai-command-r-plus", revision="pr/46")

def internet_search(query: str):
    """Returns a list of relevant document snippets for a textual query retrieved from the internet

    Args:
        query: Query to search the internet with
    """
    pass

def directly_answer():
    """Calls a standard (un-augmented) AI chatbot to generate a response given the conversation history
    """
    pass

conversation = [
    {"role": "user", "content": "Whats the biggest penguin in the world?"}
]
tools = [internet_search, directly_answer]

new_tool_use_prompt = tokenizer.apply_chat_template(
    conversation,
    tools=tools,
    chat_template="tool_use",
    tokenize=False,
    add_generation_prompt=True,
)
print(new_tool_use_prompt)