File size: 6,347 Bytes
94c8565 b444975 0efd97c d1893ac b444975 0efd97c b444975 d1893ac 26d971a 0efd97c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 |
---
license: apache-2.0
datasets:
- Team-ACE/ToolACE
language:
- en
metrics:
- accuracy
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
tags:
- code
---
# ToolACE-8B
ToolACE-8B is a finetuned model of LLaMA-3.1-8B-Instruct with our dataset [ToolACE](https://huggingface.co/datasets/Team-ACE/ToolACE) tailored for tool usage.
ToolACE-8B achieves state-of-the-art performance on the [Berkeley Function-Calling Leaderboard(BFCL)](https://gorilla.cs.berkeley.edu/leaderboard.html#leaderboard), rivaling the latest GPT-4 models.
ToolACE is an automatic agentic pipeline designed to generate **A**ccurate, **C**omplex, and div**E**rse tool-learning data.
ToolACE leverages a novel self-evolution synthesis process to curate a comprehensive API pool of 26,507 diverse APIs.
Dialogs are further generated through the interplay among multiple agents, guided by a formalized thinking process.
To ensure data accuracy, we implement a dual-layer verification system combining rule-based and model-based checks.
More details can be found in our paper on arxiv: [*ToolACE: Winning the Points of LLM Function Calling*](https://arxiv.org/abs/2409.00920)
Here are the winning scores of ToolACE in [BFCL-v3]((https://gorilla.cs.berkeley.edu/leaderboard.html#leaderboard)).
### Usage
Here we provide a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate function calling with given functions.
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "/home/huangxu/work/OpenLLMs/ToolACE-8B-zh-v2.2_1ep"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype='auto',
device_map='auto'
)
# You can modify the prompt for your task
system_prompt = """You are an expert in composing functions. You are given a question and a set of possible functions. Based on the question, you will need to make one or more function/tool calls to achieve the purpose.
If none of the function can be used, point it out. If the given question lacks the parameters required by the function, also point it out.
You should only return the function call in tools call sections.
If you decide to invoke any of the function(s), you MUST put it in the format of [func_name1(params_name1=params_value1, params_name2=params_value2...), func_name2(params)]
You SHOULD NOT include any other text in the response.
Here is a list of functions in JSON format that you can invoke.\n{functions}\n
"""
# User query
query = "Find me the sales growth rate for company XYZ for the last 3 years and also the interest coverage ratio for the same duration."
# Availabel tools in JSON format (OpenAI-format)
tools = [
{
"name": "financial_ratios.interest_coverage", "description": "Calculate a company's interest coverage ratio given the company name and duration",
"arguments": {
"type": "dict",
"properties": {
"company_name": {
"type": "string",
"description": "The name of the company."
},
"years": {
"type": "integer",
"description": "Number of past years to calculate the ratio."
}
},
"required": ["company_name", "years"]
}
},
{
"name": "sales_growth.calculate",
"description": "Calculate a company's sales growth rate given the company name and duration",
"arguments": {
"type": "dict",
"properties": {
"company": {
"type": "string",
"description": "The company that you want to get the sales growth rate for."
},
"years": {
"type": "integer",
"description": "Number of past years for which to calculate the sales growth rate."
}
},
"required": ["company", "years"]
}
},
{
"name": "weather_forecast",
"description": "Retrieve a weather forecast for a specific location and time frame.",
"arguments": {
"type": "dict",
"properties": {
"location": {
"type": "string",
"description": "The city that you want to get the weather for."
},
"days": {
"type": "integer",
"description": "Number of days for the forecast."
}
},
"required": ["location", "days"]
}
}
]
messages = [
{'role': 'system', 'content': system_prompt.format(functions=tools)},
{'role': 'user', 'content': query}
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=512, do_sample=False, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=True))
```
Then you should be able to see the following output functional calls:
```
[sales_growth.calculate(company="XYZ", years=3), financial_ratios.interest_coverage(company_name="XYZ", years=3)]
```
### Citation
If you think ToolACE is useful in your work, please cite our paper:
```
@misc{liu2024toolacewinningpointsllm,
title={ToolACE: Winning the Points of LLM Function Calling},
author={Weiwen Liu and Xu Huang and Xingshan Zeng and Xinlong Hao and Shuai Yu and Dexun Li and Shuai Wang and Weinan Gan and Zhengying Liu and Yuanqing Yu and Zezhong Wang and Yuxian Wang and Wu Ning and Yutai Hou and Bin Wang and Chuhan Wu and Xinzhi Wang and Yong Liu and Yasheng Wang and Duyu Tang and Dandan Tu and Lifeng Shang and Xin Jiang and Ruiming Tang and Defu Lian and Qun Liu and Enhong Chen},
year={2024},
eprint={2409.00920},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2409.00920},
}
```
|