|
--- |
|
license: cc-by-sa-4.0 |
|
datasets: |
|
- glaiveai/glaive-function-calling |
|
--- |
|
# glaive-function-calling-v1 |
|
|
|
glaive-function-calling-v1 is a 2.7B parameter open source chat model trained on data generated from Glaive’s synthetic data generation platform, which has similar function calling abilities as gpt-3.5 and gpt 4. |
|
|
|
The model is capable of having multi-turn conversations and intelligently choosing when to execute a function (provided at the beginning of the conversation as a system prompt) based on the conversation. The model is trained on top of the https://huggingface.co/replit/replit-code-v1-3b model. |
|
|
|
|
|
## Usage: |
|
|
|
You can run the model in the following way- |
|
``` |
|
from transformers import AutoModelForCausalLM , AutoTokenizer |
|
|
|
tokenizer = AutoTokenizer.from_pretrained("glaiveai/glaive-function-calling-v1", trust_remote_code=True) |
|
model = AutoModelForCausalLM.from_pretrained("glaiveai/glaive-function-calling-v1", trust_remote_code=True).half().cuda() |
|
|
|
inputs = tokenizer(prompt,return_tensors="pt").to(model.device) |
|
|
|
outputs = model.generate(**inputs,do_sample=True,temperature=0.1,top_p=0.95,max_new_tokens=100) |
|
|
|
print(tokenizer.decode(outputs[0],skip_special_tokens=True)) |
|
``` |
|
|
|
This model uses the following prompt format- |
|
|
|
``` |
|
SYSTEM: You are an helpful assistant who has access to the following functions to help the user, you can use the functions if needed- |
|
{ |
|
"name": "plan_holiday", |
|
"description": "Plan a holiday based on user's interests", |
|
"parameters": { |
|
"type": "object", |
|
"properties": { |
|
"destination": { |
|
"type": "string", |
|
"description": "The destination of the holiday", |
|
}, |
|
"duration": { |
|
"type": "integer", |
|
"description": "The duration of the trip in holiday", |
|
}, |
|
}, |
|
"required": ["destination", "duration"], |
|
}, |
|
} |
|
USER: I am thinking of having a 10 day long vacation in Greece, can you help me plan it? |
|
``` |
|
Based on which the model outputs- |
|
``` |
|
ASSISTANT: <functioncall> {"name": "plan_holiday", "arguments": '{ |
|
"destination": "Greece", |
|
"duration": 10 |
|
}'} |
|
``` |
|
The model precedes all function invocations with `<functioncall>`. |
|
|
|
The response of the function call should be sent to the model as- |
|
|
|
``` |
|
FUNCTION CALL: {"places_to_visit":["Athens","Santorini","Mykonos"]} |
|
``` |
|
|
|
The model can do multi-turn conversation in the above format. |
|
|
|
|
|
We're working on providing an inference server which can act as a drop in replacement to the OpenAI API, you can follow [this](https://github.com/glaive-ai/function-calling-server) repo for the server. |
|
|
|
## Known Limitations: |
|
|
|
- While the model does well on function calling use-cases, it doesn't always generalize very well to other chat use-cases. This is intentional as our thesis at Glaive is to provide use-case specialised model that are only used for the given task. |
|
- The model may sometimes hallucinate functions, v2 of the model will be aimed to fix that with a bigger dataset. |
