PLAY 2 PROMPT : Zero-shot Tool Instruction Optimization for LLM Agents via Tool Play (2025)

Abstract: Large language models (LLMs) are increasingly integrated with specialized external tools, yet many tasks demand zero-shot tool usage with minimal or noisy documentation. Existing solutions rely on manual rewriting or labeled data for validation, making them inapplicable in true zero-shot settings. To address these challenges, we propose PLAY2PROMPT, an automated framework that systematically “plays” with each tool to explore its input-output behaviors. Through this iterative trial-and-error process, PLAY2PROMPT refines tool documentation and generates usage examples without any labeled data. These examples not only guide LLM inference but also serve as validation to further enhance tool utilization. Extensive experiments on real-world tasks demonstrate that PLAY2PROMPT significantly improves zero-shot tool performance across both open and closed models, offering a scalable and effective solution for domain-specific tool integration.

## introduction recently, there has been growing research interest in agentic large language model (llm) frameworks, where, rather than having llms answer requests and queries from their own knowledge, llms can call a set of external tools with specialized capabilities. this allows llms to address more complex tasks and produce responses more accurately @xcite . one key challenge in developing agentic llm frameworks is how to dynamically learn to use new, user-defined tools, which is crucial because having a fixed, general-purpose tool set is often insufficient for real-world scenarios requiring domain-specific functionalities. the existing mainstream paradigm for dynamically incorporating new tools is by supplementing user-defined tools at inference time in a zero-or few-shot manner via prompting @xcite , leveraging zero-shot tool-calling capabilities of current llms that have been tuned with tool-use instructions. the success of this paradigm depends on providing sufficient information about the new tool in the prompt. specifically, existing approaches generally rely on two types of information: ❶ comprehensive tool documentation detailing the tool's functionalities and input/output formats, and ❷ in-context demonstrations that include example queries and corresponding tool calls @xcite . inadequate documentation can lead to failures in tool usage, such as syntax errors in both zero-shot and fine-tuned models @xcite , hallucinations due to incomplete or incorrect tool documentation @xcite , and diminished performance resulting from inadequate demonstrations @xcite . however, in many practical scenarios, it is often not realistic to rely on users, who are often non-experts in ai, to provide adequate documentation for their tools, nor to craft tool-use examples. when users do provide documentation, it may lack crucial details needed for llms to call the tools correctly. while automatic prompt optimization techniques @xcite could enhance tool documentation, they still require sufficient tool-use examples, which are unavailable in true zero-shot settings. in short, without tool-use examples, neither polished documentation nor in-context demonstrations can be supplied, leading to significant performance degradation in integrating new tools. to address these challenges in zero-shot tool utilization, we introduce play2prompt, an automated framework that generates both high-quality tool documentation and tool-use demonstrations, as illustrated in figure 1 . unlike prior works, play2prompt does not rely on any external labeled examples. instead, it systematically interacts with the new tools-mimicking human trial-anderror-and observes both successful and failed attempts to gather evidence about each tool's correct usage. using insights from this "tool-play", play2prompt creates example demonstrations and refines the tool documentation that better guide llms in subsequent inference. play2prompt consists of two steps. in step 1, a set of tool-use examples are generated via a trial-and-error process, where an llm agent iteratively call the target tool with different invocation parameters until correct invocation is found. then, for each correct tool invocation instance, a query is generated such that it can be answered by the tool invocation, forming a question-answer pair as a tool-use example. in step 2, the tool documentation are refined, using the generated tool-use examples as a validation set. in both steps, we employ self-reflection @xcite @xcite to generate error feedback, thereby directing the search algorithm towards progressively improved outputs. because play2prompt operates entirely in a zero-shot manner and is inherently task-agnostic, it offers a practical and scalable solution for enhancing llm tool utilization without additional labeled data or human intervention. we evaluate play2prompt with benchmark on real-world scenarios. on the berkeley function-calling leaderboard @xcite and the sta-bletoolbench benchmark @xcite , our approach consistently surpasses baseline methods for both open @xcite @xcite and closed models @xcite . extensive experiments and analyses further underscore the robustness of our approach. our contributions can be summarized as follows: ## related work llms for tool use. recent years have seen notable advances in employing large language models (llms) as agents to master tool use for solving complex tasks @xcite , thereby enhancing llms' capabilities in multi-modal understanding @xcite @xcite , programming tools @xcite @xcite @xcite , and other domain-specific functionalities. the conventional approach involves training base models with tool-use data @xcite or fine-tuning llms @xcite @xcite @xcite , but may require continual learning as new tools are added. @xcite addressed scalability by training tool embeddings plug-and-play usage, though still requiring labeled data. alternatively, llms can be augmented with meta-prompts or tool-use instructions at inference time @xcite @xcite @xcite . as the range of applications and tools expands, enhancing llms' capacity to handle new tools remain pivotal, which we address through play2prompt. tool-use instructions and optimization. tool documentation and example demonstrations are key to prompting llms for effective tool use. hsieh et al. (2023) highlighted the risk of hallucinations when documentation is lacking, and xu et al. (2023) showed performance declines without in-context examples. to automate generating tool-use instances, shen et al. (2024) leveraged a graph of tool relations for back-instructing queries, relying on the availability of external tool graphs. yuan et al. (2024) proposed direct prompting to rewrite tool documentation, which relies on labeled documentation examples and lacks systematic optimization and the ability to measure optimization quality. qu et al. (2025) explored llms interacting with tools, focusing on tool documentation only with single-thread iterative refinement. automatic prompt tuning @xcite adapts prompts to domain-specific tasks but require held-out test sets, rendering it unsuitable for zero-shot tool instruction rewriting @xcite . these constraints underscore the need for approaches that optimize tool instructions and demonstrations without labeled data or manual effort, which play2prompt achieves by interacting directly with the tool itself. ## conclusion we present play2prompt, an automated framework that iteratively refines tool documentation and creates usage demonstrations, enhancing llms' tool use in zero-shot settings. through a searchbased trial-and-error process with self-reflection, play2prompt allows models to explore and improve tool use without labeled data or extensive manual effort. this approach addresses the limitations of methods that rely on handcrafted prompts or labeled data, providing a scalable and taskagnostic solution for real-world applications.