APIGen: Automated Pipeline for Generating Verifiable and Diverse
  Function-Calling Datasets
Authors: name: Zuxin Liu; name: Thai Hoang; name: Jianguo Zhang; name: Ming Zhu; name: Tian Lan; name: Shirley Kokane; name: Juntao Tan; name: Weiran Yao; name: Zhiwei Liu; name: Yihao Feng; name: Rithesh Murthy; name: Liangwei Yang; name: Silvio Savarese; name: Juan Carlos Niebles; name: Huan Wang; name: Shelby Heinecke; name: Caiming Xiong
Published: 2024-06-26 17:49:11

Abstract: The advancement of function-calling agent models requires diverse, reliable,
and high-quality datasets. This paper presents APIGen, an automated data
generation pipeline designed to synthesize verifiable high-quality datasets for
function-calling applications. We leverage APIGen and collect 3,673 executable
APIs across 21 different categories to generate diverse function-calling
datasets in a scalable and structured manner. Each data in our dataset is
verified through three hierarchical stages: format checking, actual function
executions, and semantic verification, ensuring its reliability and
correctness. We demonstrate that models trained with our curated datasets, even
with only 7B parameters, can achieve state-of-the-art performance on the
Berkeley Function-Calling Benchmark, outperforming multiple GPT-4 models.
Moreover, our 1B model achieves exceptional performance, surpassing
GPT-3.5-Turbo and Claude-3 Haiku. We release a dataset containing 60,000
high-quality entries, aiming to advance the field of function-calling agent
domains. The dataset is available on Huggingface:
https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k and the
project homepage: https://apigen-pipeline.github.io/