Quality Matters: Evaluating Synthetic Data for Tool-Using LLM s (2024)

Abstract: Training large language models (LLMs) for external tool usage is a rapidly expanding field, with recent research focusing on generating synthetic data to address the shortage of available data. However, the absence of systematic data quality checks poses complications for properly training and testing models. To that end, we propose two approaches for assessing the reliability of data for training LLMs to use external tools. The first approach uses intuitive, human-defined correctness criteria. The second approach uses a model-driven assessment with in-context evaluation. We conduct a thorough evaluation of data quality on two popular benchmarks, followed by an extrinsic evaluation that showcases the impact of data quality on model performance. Our results demonstrate that models trained on high-quality data outperform those trained on unvalidated data, even when trained with a smaller quantity of data. These findings empirically support the significance of assessing and ensuring the reliability of training data for tool-using LLMs.

## introduction enabling llms to make use of external tools is a promising frontier that allows tapping into information that is not readily available to the model itself @xcite @xcite @xcite @xcite . given a request and a list of available external api functions, the basic task of a model is to collect information by invoking functions, and then to generate a response for the request. due to the lack of data for the task and the high cost of creating such data, researchers have devised synthetic datasets, predominantly with the assistance of llms @xcite @xcite . these datasets have facilitated a great leap in promoting the appealing applications of tool-using llms. recently, @xcite showed that higher quality training data yields better performance by llms in text generation tasks. however, leading works on tool-using llms have not made an effort to measure the quality of training data. rather, only model outputs are extrinsically evaluated, disregarding the effect of the data on the tested models. most research on tool-using llms focuses on improving training and evaluation processes @xcite @xcite . the lack of attention to data quality makes it difficult to interpret potential pitfalls for models. in turn, this wastes valuable resources for configuring and tuning models over possibly erroneous data. datasets for tool-using llms comprise instructions and ground truth api call sequences, and are created mainly with llms. two such prominent datasets @xcite were produced with the help of chatgpt (openai, 2024), and were not explicitly assessed for their quality. a closer inspection, conducted in this work, reveals numerous errors within the data, both in the instructions and in the ground-truth api calls ( §4). to conduct our inspections, we define intrinsic measures for data quality assessment, focusing on different aspects of quality. for each aspect we outline human evaluation guidelines, as well as implement automated methods for evaluation. the automatic methods employ chatgpt, either by directly asking for its evaluation or by having it perform a proxy task and deriving the evaluation from its output. we show high agreement for our automated methods with expert human annotations. in addition to the intrinsic measures, we propose a metric we call in-context evaluation (ice; §5). ice evaluates a data instance by how helpful it is for in-context learning, thus predicting its helpfulness for training a model ( §5). this metric is fully automated and does not rely on task-specific measurement definitions. other than being appraisal instruments, the intrinsic evaluation and ice metrics can be used to automatically filter out low quality data from an existing dataset. in section 6 we carry out this procedure, and display the effect of training toolusing llms with higher quality data. our findings, demonstrated on the toolbench @xcite and toolalpaca @xcite benchmarks, show either better or comparable performance when using a small high-quality training dataset, compared to the original models trained on larger unverified datasets. the two benchmarks are based on different api function sets, and different data generation and training methods, indicating the generalized applicability of our methods. ## task setup tool-using llms are expected to behave as follows. given a set of tools @xmath0, ..., t n }, represented as api functions, and an instruction query q, a model is required to plan a call sequence @xmath1, ..., t ′ k ), based on t , that would obtain information, or perform actions, needed to address q. based on the responses obtained after performing the call sequence (using an external api invoker), the model then generates a final response r that responds to q. the primary method for model evaluation is based on calculating the pass rate, which measures the proportion of instances that successfully addressed their instructions, i.e., a predicted r responded to q adequately (explained further in §6). as mentioned in section 2, the prominent datasets created for training and testing tool-using models were created synthetically with the assistance of llms. specifically, we utilize the tool-bench @xcite and toolalpaca @xcite datasets. ## in-context evaluation (ice) as an alternative data measurement using intrinsic evaluation, we have defined an intuitive and straightforward approach to identify lowquality data instances based on human understanding of data correctness. however, assessing the "educational" value of an instance, i.e., its contribution to the learning process of a model, is a complex task. in addition, the intrinsic evaluation metrics proposed rely on prompting a powerful llm, which can become costly on large datasets. to address these challenges, we propose in-context evaluation (ice) as an alternative automatic approach for assessing data quality. recent studies found a connection between incontext learning and fine-tuning, demonstrating that language models implicitly perform gradient descent when dealing with in-context tasks @xcite . motivated by this insight, we seek to evaluate the educational value of each data instance by measuring the performance of in-context learning using the specific instance as a one-shot example. ## extrinsic evaluation in this section, we validate our main claim that finetuning a tool-using llm with a smaller dataset of high-quality data can lead to better performance of the model on the task, compared to a larger noisy dataset. we use both intrinsic metrics and ice to create training sets of varying quality, and compare the results of training with the different sets. training setup. we follow the general setup used by the toolalpaca @xcite and toolbench @xcite benchmarks. specifically, we fine-tune vicuna-7b @xcite for toolalpaca, and llama-7b @xcite for toolbench, both using lora (hu et al., 2022) (see appendix c.1 for more details). we use the following train sub-sets from each model's respective benchmark training sets: each fine-tuned model is evaluated using pass rate, which is an extrinsic evaluation procedure used in both benchmarks. 8 this measures the proportion of instances in which the resulting api-call sequences and responses adequately address their respective instruction query. see appendix c.2 for more details on the evaluation procedure. test sets. for toolalpaca we use the original test set, as it is created with human annotation. it consists of 100 instructions of simulated tools that were not part of the training tool set. toolbench test set was created using llms and was not manually validated. we inspected 674 examples, as detailed in appendix a.4. for instances of low quality, we either rectified them (e.g., manually adding a missing parameter value), or discarded them. the resulting test set contains 420 high-quality examples. foot_6 ## conclusion we demonstrated the importance of evaluating the quality of training data for fine-tuning tool-using llms. we introduce two data-evaluation approaches. the first is a rigorously devised intrinsic quality assessment, for which we implement automated metrics. the second uses in-context evaluation, that measures the educational value of training examples. while the former method is more explainable and dependable, the latter is computationally cheaper. we apply both approaches to filter data instances from two large datasets of differing qualities. the resulting subsets of training data demonstrate comparable or superior quality in terms of model performance, despite their smaller size compared to the original datasets. overall, we observe that it is worthwhile to more carefully choose the training data for tool-using llms. if investing in better methods of data generation is costly, automatic post-hoc filtration can be a great alternative.