Enhancing Tool Retrieval with Iterative Feedback from Large Language Models (2024)

Abstract: Tool learning aims to enhance and expand large language models’ (LLMs) capabilities with external tools, which has gained significant attention recently. Current methods have shown that LLMs can effectively handle a certain amount of tools through in-context learning or fine-tuning. However, in real-world scenarios, the number of tools is typically extensive and irregularly updated, emphasizing the necessity for a dedicated tool retrieval component. Tool retrieval is nontrivial due to the following challenges: 1) complex user instructions and tool descriptions; 2) misalignment between tool retrieval and tool usage models. To address the above issues, we propose to enhance tool retrieval with iterative feedback from the large language model. Specifically, we prompt the tool usage model, i.e., the LLM, to provide feedback for the tool retriever model in multi-round, which could progressively improve the tool retriever’s understanding of instructions and tools and reduce the gap between the two standalone components. We build a unified and comprehensive benchmark to evaluate tool retrieval models. The extensive experiments indicate that our proposed approach achieves advanced performance in both in-domain evaluation and out-of-domain evaluation.

## introduction large language models (llms) have demonstrated remarkable success in language-related tasks and are considered a potential pathway to achieving artificial general intelligence @xcite . however, despite their powerful capabilities, llms are still limited in many aspects, such as knowledge update and mathematical reasoning. a promising way to overcome these limitations is to empower llms with external tools, known as tool † corresponding author. 1 code available at https://github.com/travis-xu/trfeedback . the challenges posed by the extensive and frequently updated tools require the external tool retrieval component. ## conclusion and future work in this study, we concentrate on the crucial tool retrieval in the tool learning of llms. we have identified the bottleneck in the tool retrieval-usage pipeline as the limited tool retrieval model. we propose the unique challenges of the tool retrieval compared with document retrieval. to improve the current tool retrieval process, we propose leveraging the llm's feedback to assess the retrieval results and provide detailed suggestions for refining user instructions. in order to integrate the retriever model into this iterative process, we implement iteration-aware feedback training. this will improve the tool retriever's capabilities and close the gap between tool retrieval and usage models. we conduct the tr-benchmark to comprehensively evaluate the models' ability in real-world tool retrieval scenarios. our method demonstrates the best performance in both in-domain and out-of-domain settings. in the future, we aim to improve this work from the following aspects. 1) limited by the training speed, we have applied the offline feedback generation, where feedback is generated before training the tool retriever. we will also assess whether online feedback generation yields further improvements in the future. 2) furthermore, as the tool retriever serves the subsequent tool usage model in tool learning, we intend to conduct further evaluations of the tool retriever models based on the subsequent tool usage results. limitations 1) undoubtedly, our iterative refinement will reduce the inference speed of the tool retrieval. the efficiency issue is inherent in approaches involving llms' interaction. we have evaluated the efficiency as the number of iterative rounds increases. fortunately, we observed that the retrieval model can achieve a significant performance improvement after just a single round of llms' feedback compared to without feedback. furthermore, the performance enhancement of the tool retrieval is crucial for the subsequent tool usage model, ensuring that the correct tools are retrieved and lays the foundation for all subsequent steps of tool usage. therefore, we believe that performance improvement is worthwhile despite some efficiency loss. we will also pay more attention to this issue in the future.