Divide, Optimize, Merge: Scalable Fine-Grained Generative Optimization for LLM Agents (2025)

Abstract: LLM-based optimization has shown remarkable potential in improving agentic systems. However, the conventional approach of prompting LLM-based generative optimizer with the trajectories on the whole training dataset in a single pass becomes untenable as datasets grow, leading to context window overflow and degraded pattern recognition. To address these challenges, we propose Fine-grained Generative Optimization (FGO), a scalable framework that divides large optimization tasks into manageable subsets, performs targeted optimizations, and systematically combines optimized components through progressive merging.Evaluation across ALFWorld, LogisticsQA, and GAIA benchmarks demonstrates that FGO outperforms conventional approach by 1.6-8.6% while reducing average prompt token consumption by 56.3%. Our framework provides a practical solution for scaling up LLM-based generative optimization of increasingly sophisticated agentic systems. Further analysis demonstrates that FGO achieves the most consistent performance gain in all training dataset sizes, showcasing its scalability and efficiency.

## introduction large language models (llms) have emerged as powerful optimizers for agentic systems, capable of analyzing execution trajectories and refining system modules like prompts @xcite @xcite @xcite , tools @xcite @xcite . this paradigm, termed generative optimization @xcite , leverages the llm's inherent ability to comprehend and learn from execution patterns, allowing for systematic improvements without requiring access to model weights or gradients. the optimized agentic system has shown great potential in various domains, including reasoning @xcite 2024; @xcite , software engineering @xcite @xcite , data analysis @xcite , computer operation @xcite @xcite . however, due to the increasing volume of data required for optimizing llm agentic systems autonomously, directly applying llm-based optimization approaches encounters a fundamental scalability issue. existing methods typically concatenate all execution trajectories on the training data and perform optimization in an all-at-once manner, feeding the entire dataset into the llm optimizer in a single prompt. while this approach works for optimization tasks with small-scale data, it becomes problematic as the data grows. for instance, in the gaia benchmark @xcite , agents are tasked to retrieve and analyze information from a diverse range of sources, including web and different types of files. solving a task from the dataset requires agents to interact with external tools or forge their own ones in order to collect real-world information and perform reasoning. the sophisticated process can generate lengthy execution traces for subsequent optimization, which is filled with raw documents and complex intermediate reasoning steps, even challenging for human to parse @xcite . this increasing complexity leads to two critical limitations: (1) the concatenated trajectories exceed llm context windows, forcing truncation of valuable optimization signals. (2) even when content fits within context windows, llms struggle with analyzing long-range dependencies in extensive corpus @xcite @xcite @xcite , making it hard for the llm optimizer to capture subtle patterns and relationships between execution traces. as a result, such approaches can produce suboptimal solutions, particularly in complex scenarios where understanding the intricate relationships between different execution trajectories is crucial for improving agent performance. to address these scalability challenges, we introduce fine-grained generative optimization (fgo), a framework that enables efficient optimization of llm-based agentic systems with large-scale data. specifically, fgo operates through three components: (1) task division that breaks down the large training dataset into more manageable subsets, (2) fine-grained generative optimization enabling efficient processing of each subset, and (3) progressive module merging that adaptively combines the individually optimized modules while preserving crucial insights from each subset. this design allows fgo to effectively handle larger optimization tasks while maintaining high-quality results. we evaluate fgo by optimizing two agent modules: instruction prompts and tools an agent could access, on diverse tasks including alf-world @xcite , logisticsqa, and gaia @xcite . agents trained with fgo produce significant performance gains across all datasets, ranging from 8.3% to 38.1%, outperforming other optimization methods by 1.6%-8.6%. further analysis reveals that fgo maintains superior performance across varying training dataset sizes, highlighting its scalability and stability. notably, fgo achieves these improvements while reducing prompt token consumption by 56.3% and increasing optimization efficiency by 7.6% compared to conventional all-at-once optimization. our contributions are threefold: (1) we identify and analyze the scalability limitations in current llm-based optimization approaches for agentic systems. (2) to address the scalability limitation, we propose fgo, a scalable optimization framework that effectively handles large-scale agent optimization through task division and progressive merging. (3) we conduct extensive experiments across diverse tasks to validate fgo's effectiveness and provide insights into its scalability advantages through comprehensive empirical analysis. ## conclusion in this paper, we address the scalability challenges in llm-based agent optimization by introducing fgo, a framework that optimizes system modules through task division, fine-grained targeted optimization, and progressive merging. evaluation across multiple benchmarks demonstrates that fgo consistently outperforms existing optimization approaches. analysis shows that fgo achieves an optimal balance between performance and cost. fgo's scalability makes it valuable for optimizing sophisticated agentic systems in real-world cases.