SauravP97/tooling-gemma-270M-inst

Introduction

Tooling Gemma is a fine-tuned version of Gemma 3 270M base model tuned for function calling. Our model will take APIs / Functions documentation (structure) as an input from the user along with user's query and will perform Function Call with right arguments wherever needed.

Note: The Functions / APIs structure is generally provided as a part of the System Instructions.

An Example User-Model communication will look like this:

SYSTEM: You are a helpful assistant with access to the following functions. Use them if required -
{
  "name": "calculate_factorial",
  "description": "Calculate the factorial of a given number",
  "parameters": {
    "type": "object",
    "properties": {
      "number": {
        "type": "integer",
        "description": "The number to calculate factorial for"
      }
    },
    "required": [
      "number"
    ],
  }
}


USER: Hi, I need to calculate the factorial of 5. Can you help me with that?

ASSISTANT: <functioncall> {"name": "calculate_factorial", "arguments": '{"number": 5}'} <|endoftext|>

FUNCTION RESPONSE: {"result": 120}

ASSISTANT: The factorial of 5 is 120. <|endoftext|>

USER: That was quick! Can you also calculate the factorial of 7?

ASSISTANT: <functioncall> {"name": "calculate_factorial", "arguments": '{"number": 7}'} <|endoftext|>

Using the model

class ToolingGemma:
    def __init__(self, system_instructions):
        self.chat_history = ''
        self.model = AutoModelForCausalLM.from_pretrained('SauravP97/tooling-gemma-270M-inst', device_map="cpu")
        self.tokenizer = AutoTokenizer.from_pretrained('google/gemma-3-270m')
        self.system_instructions = system_instructions
        self.stop_word = "<|endoftext|>"

    def generate(self, user_query):
        user_query = 'USER: ' + user_query
        if self.chat_history:
            prompt = self.chat_history + '\n' + user_query + '\n' + 'ASSISTANT:'
        else:
            prompt = self.system_instructions + '\n\n' + user_query + '\n' + 'ASSISTANT:'

        input_ids = self.tokenizer(prompt, return_tensors="pt")

        agent_response = self.model.generate(
            **input_ids, 
            generation_config=GenerationConfig.from_dict({"max_new_tokens": 1000}),
            stop_strings=[self.stop_word],
            tokenizer=self.tokenizer,
        )

        decoded_agent_response = self.tokenizer.decode(agent_response[0])
        self.chat_history = decoded_agent_response

        return decoded_agent_response

system_instructions = '''
SYSTEM: You are a helpful assistant with access to the following functions. Use them if required -
{
    "name": "calculate_discount",
    "description": "Calculate the discounted price of a product",
    "parameters": {
        "type": "object",
        "properties": {
            "original_price": {
                "type": "number",
                "description": "The original price of the product"
            },
            "discount_percentage": {
                "type": "number",
                "description": "The discount percentage"
            }
        },
        "required": [
            "original_price",
            "discount_percentage"
        ]
    }
}
'''

tooling_gemma_model = ToolingGemma(system_instructions=system_instructions)

agent_response = tooling_gemma_model.generate('Can you please book a flight for me from New York to London?')

Model Output:

ASSISTANT: I'm sorry, but I'm unable to assist with booking flights. My current capabilities are limited to calculating discounted prices based on original price and discount percentage. If you need help with that, feel free to ask! <|endoftext|>

Continue talking to the model:

agent_response = tooling_gemma_model.generate('Calculate the discounted price for 100 dollars at a discount of 30%')

Model Output:

ASSISTANT: <functioncall> {"name": "calculate_discount", "arguments": '{"original_price": 100, "discount_percentage": 30}'} <|endoftext|>