LAM SIMULATOR : Advancing Data Generation for Large Action Model Training via Online Exploration and Trajectory Feedback (2025)

Abstract: Large Action Models (LAMs) for AI Agents offer incredible potential but face challenges due to the need for high-quality training data, especially for multi-steps tasks that involve planning, executing tool calls, and responding to feedback. To address these issues, we present LAM SIMULATOR, a comprehensive framework designed for online exploration of agentic tasks with high-quality feedback. Our framework features a dynamic task query generator, an extensive collection of tools, and an interactive environment where Large Language Model (LLM) Agents can call tools and receive real-time feedback. This setup enables LLM Agents to explore and solve tasks autonomously, facilitating the discovery of multiple approaches to tackle any given task. The resulting action trajectory data are then used to create high-quality training datasets for LAMs. Our experiments on popular agentic benchmarks, ToolBench and CRMArena, highlight the effectiveness of LAM SIMULATOR: models trained with self-generated datasets using our framework achieve significant performance gains, up to a 49.3% improvement over their original baselines. LAM SIMULATOR requires minimal human input during dataset creation, highlighting LAM SIMULATOR’s efficiency and effectiveness in speeding up development of AI agents.

## introduction large action models (lams) @xcite @xcite are an advanced type of large language model, specifically optimized for tool usage, reasoning, and function calling. recent advancements have propelled their capabilities, making them integral to applications such as ai agents and task automation. lams benefit from specialized training for enhanced performance in agent applications. as use cases grow, * core contributors. current approaches for creating lams include prompt engineering, incorporating additional contextual information into prompts, supervised fine-tuning (sft), reinforcement learning from human feedback (rlhf) @xcite , among others. most of these methods, however, rely heavily on manual data curation, a process that is both time-consuming and expensive. to address this challenge, the use of llm agents to explore environments autonomously has emerged as a promising method to reduce the need for human labeling and annotation in agent model development. recent studies, such as tooltalk @xcite , webarena @xcite , apigen @xcite , and learn-by-interact @xcite , have demonstrated the ability to automatically generate action trajectory data for agent learning and evaluation. however, tooltalk is limited to specific tasks that are curated or filtered by humans. webarena offers a very limited action space within web domain. apigen and learn-by-interact, while showing considerable potential in generating agentic data, are limited to the heavy usage of llms to assess data quality, thus introducing a notable amount of uncertainty, which is an important issue to consider. given these limitations, we introduce lam sim-ulator, a comprehensive framework designed to enhance data generation for agent learning through exploration. as shown in figure 1 , lam simula-tor employs a template-filling strategy to dynamically create queries. users only need to develop a series of query templates and descriptions for query parameters, along with logic to compute task answers using these parameters. a large language model (llm) then creates values to fill into the query parameters in real-time and populates the templates to form real user queries. these generated queries are then given to llm agents, which query instance generation trajectory synthesis yes llm agent no is f inal action? paraphrase {"[time_period]": "5 months"} randomly sampled variable available tools get _s t ar t _dat e get _c as es get _i s s ues in the past @xcite , find the agent with the ... ## related work with the rapid evolution of large language models (llms), there has been a significant increase in their application to tool-use and functioncalling scenarios. enhancing the capabilities of llms @xcite @xcite with external tools allows them to go beyond the limitations of their static parametric knowledge and text-based input-output interfaces. this extension enables them to access real-time information, leverage external reasoning systems, and perform meaningful actions in dynamic environments. recently, open-sourced research has focused increasingly on enhancing the efficiency of llms in tool-use contexts @xcite @xcite @xcite , while also exploring various prompting and training strategies to improve their performance in agentic tasks. prominent prompting techniques like chain of thought (cot) @xcite , reflection @xcite , and react @xcite have garnered attention. while initial efforts centered on in-context learning (icl)-where pre-trained llms were prompted with api specifications and tool-use examples-current approaches are increasingly incorporating fine-tuning methods to enhance model accuracy. moreover, popular agent environments such as tooleval @xcite , agentbench @xcite , webarena @xcite , osworld @xcite , agentboard @xcite , spectool @xcite , and τbench @xcite facilitate agent interactions and evaluations within various scenarios such as web navigation, shopping, games, and computer environments. tooleval offers a large test set with diverse scenarios, but the heavy reliance on llms for evaluation introduces undesired and uncontrollable variability. agentboard designs two multi-turn environments for tool query and operations, but they only contain 100 user queries in total and do not include real-time feedback. webarena, osworld, and τ -bench simulates conversations between a user and a language agent, providing api tools and policy guidelines with some limited domains: as web, os, retail and airline. the development of data generation pipelines for agentic learning has also seen significant advancements in recent years. toolbench @xcite introduced a wide range of tasks and tools but often suffers from lower data quality because it relies heavily on llms to create and evaluate tasks, which can introduce noise. apigen @xcite improves data quality by using multilayered evaluation schemes, but it is limited to handling only single-turn conversations. eto @xcite uses a fixed set of instructions and labeled data to generate trajectories, making it unsuitable for situations lacking pre-existing training data. lastly, learn-by-interact @xcite brings diversity to tasks by constructing them backward from desired outcomes, but its dependence on comprehensive documentation limits its usefulness in domains with sparse references. our proposed framework, lam simulator, addresses these limitations and introduces several key innovations. unlike previous approaches, lam simulator natively supports complex multi-turn conversations, rather than being confined to singleturn interactions. it also allows an open-ended action space with limited context or documentation needed, meaning the agent can use any combination of tools without restrictions. we automate data generation at scale, combining a vast collection of tools with carefully-crafted, high-quality task templates. furthermore, all evaluations in our framework are fully programmatic, ensuring the consistency and accuracy of the constructed datasets by avoiding the variability introduced by llm-based evaluation. ## lam simulator we propose lam simulator to enable llm agents to autonomously explore and enhance their problem-solving skills. we first construct query instances, each of which details the goals the llm agents should accomplish and the available tools ( §3.2). based on the given query instance, llm agents self-synthesize trajectories by iteratively interacting with the environment until the final state is reached ( §3.3). finally, the generated trajectories are filtered based on ground-truth answers, and subsequently used for training llm agents. figure 1 shows an overview of our framework. we describe how each component works in the following subsections. ## applications lam simulator is engineered to be seamlessly adaptable to a wide range of scenarios, showcasing its potential in advancing contemporary models. to demonstrate the effectiveness of lam simula-tor, we supported its application to two prominent environments for agentic tasks: toolbench @xcite and crmarena @xcite . toolbench emphasizes generic tool-use capability, while crmarena delves into complex and realistic customer relationship management (crm) scenarios. our goal is to exhibit llm agent's selfimprovement capability via explorative processes through lam simulator. ## experimental setup with the integration of environments into lam simulator detailed in §4, we conducted experiments to demonstrate our framework's effectiveness on toolbench and crmarena benchmarks. ## conclusion in this paper, we presented lam simulator, a comprehensive framework designed to advance the development of large action models (lams) by enabling self-learning through online exploration and automated prorgammatic feedback. our system effectively addresses the limitations of traditional supervised learning and manual data curation, offering a scalable solution that enhances both agentic performance and training efficiency. lam simulator provides real-time interactions, multiturn task processing, and high-quality feedback, contributing to significant improvements in model training performance across various benchmarks, such as toolbench and crmarena, where models trained with self-generated data via lam simu-lator gained a significant improvements and potentially outperformed other leading models. our framework accelerates the learning and adaptation process of lams with minimal human intervention, demonstrating its potential as a pivotal tool for future research and development in ai agents.