Mistral-7B-FunctionCalling-v0.3 🐯🔧

Overview

Mistral-7B-FunctionCalling-v0.3 is a fine-tuned version of Mistral 7B, optimized for function calling. It enables the model to interact with external tools and APIs, making it ideal for building AI agents and automated workflows. It has been trained from the base model and cannot be used for chat purpose.

Notebook

https://github.com/duranbe/finetuning-llms/blob/main/nb/FinetuningMistral7B.ipynb

Usage

Mistral Inference

from mistral_common.protocol.instruct.messages import UserMessage
from mistral_common.protocol.instruct.tool_calls import Function, Tool
from mistral_inference.transformer import Transformer
from mistral_inference.generate import generate

from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
from mistral_common.protocol.instruct.request import ChatCompletionRequest

# Or just MistralTokenizer.v3()
tokenizer = MistralTokenizer.from_file("/Mistral-7B-FunctionCalling-v0.3/tokenizer.model.v3")
model = Transformer.from_folder("/Mistral-7B-FunctionCalling-v0.3/mistral_models")

weather_tool = Tool(function=Function(
                  name="get_current_weather",
                  description="Get the current weather",
                  parameters={
                    "type": "object",
                    "properties": {
                        "city": {
                            "type": "string",
                            "description": "The city and state, e.g. San Francisco, CA",
                        },
                    },
                    "required": ["city"],
                },
            )
        )

completion_request = ChatCompletionRequest(tools=[weather_tool], messages=[UserMessage(content="Whats the current weather in the city of London, United Kingdom")])
tokens = tokenizer.encode_chat_completion(completion_request).tokens

out_tokens, _ = generate([tokens], model, max_tokens=256, temperature=0.0, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)
result = tokenizer.instruct_tokenizer.tokenizer.decode(out_tokens[0])

Hugging Face Transformers

TBA