MPO : Boosting LLM Agents with Meta Plan Optimization (2025)

Abstract: Recent advancements in large language models (LLMs) have enabled LLM-based agents to successfully tackle interactive planning tasks. However, despite their successes, existing approaches often suffer from planning hallucinations and require retraining for each new agent. To address these challenges, we propose the **M**eta **P**lan **O**ptimization (**MPO**) framework, , which enhances agent planning capabilities by directly incorporating explicit guidance. Unlike previous methods that rely on complex knowledge, which either require significant human effort or lack quality assurance, MPO leverages high-level general guidance through meta plans to assist agent planning and enables continuous optimization of the meta plans based on feedback from the agent’s task execution. Our experiments conducted on two representative tasks demonstrate that MPO significantly outperforms existing baselines. Moreover, our analysis indicates that MPO provides a plug-and-play solution that enhances both task completion efficiency and generalization capabilities in previous unseen scenarios.

## introduction recent advancements in large language models (llms) @xcite @xcite have enabled llm-based agents to tackle complex multi-step tasks, including embodied housework @xcite and science experiments @xcite . these tasks require sophisticated planning abilities, as agents need to understand long-term dependencies @xcite , reason about sequential actions, and adapt to dynamic environments @xcite . the planning quality of these agents plays a crucial role in determining their overall performance. current mainstream llm-based agents primarily develop their planning capabilities through implicit methods, either directly leveraging the model's inner ability or fine-tuning from expert trajectories. for example, react @xcite and reflexion @xcite perform planning on-the-fly during task execution and are prone to getting lost due to planning hallucination @xcite . the works including agenttuning @xcite , lumos @xcite , and eto @xcite employ trajectory tuning to enhance implicit planning capabilities and require retraining for each new agent, resulting in huge computational cost (figure 1(a) ). beyond implicit planning, recent studies have explored the use of human knowledge to guide agents in task execution, capitalizing on the benefits of explicit guidance and low integration costs of such knowledge @xcite . however, these approaches face significant challenges: they either require extensive manual efforts or struggle to ensure quality in the process of acquiring complex knowledge, potentially leading to inconsistent improvements in agent perfor-mance @xcite . to overcome these limitations, we propose to automatically generate a high-level abstract guidance, termed meta plan, which emulates human prior knowledge. unlike previous implicit plans derived during execution, meta plans are decoupled from specific environmental details and complex agent trajectories, reducing the difficulty of knowledge acquisition. figure 1(b) illustrates an abstract meta plan for the task "put some watch on safe". in contrast to the concrete plan in figure 1 (a), the meta plan omits fine-grained details (e.g., cabinet 4). to further enhance meta plan quality, we design a meta plan optimization (mpo) framework that iteratively improves plans based on environmental feedback. this process mirrors how humans refine their strategies through experience, ensuring that meta plans evolve over time for optimal task execution. the mpo framework comprises two key components: a meta planner and an agent. the meta planner generates high-level meta plans, while the agent provides execution feedback to evaluate the quality of the input meta plans and guide meta planner refinement. initially, we collect meta plans from expert trajectories and cold-start the meta planner through supervised fine-tuning. to further optimize the meta planner, we use monte carlo (mc) sampling to estimate the task completion rate of the agent as feedback. specifically, given a task, the planner generates multiple meta plans through sampling. then for each meta plan, the agent is also sampled to produce multiple execution trajectories, and the task completion rate is estimated accordingly. after identifying contrastive meta plan pairs-those yielding the highest and lowest task completion rates-we apply dpo @xcite to refine the meta planner on these plan pairs. finally, the trained meta planner can be detached from the mpo framework and function as a portable component, capable of generating highquality meta plans for tasks in the target environment. this facilitates task completion for any new agent without incurring additional training costs. we evaluate our approach on three representative benchmarks: alfworld @xcite for embodied household tasks, scienceworld @xcite for textual science experiment tasks and webshop @xcite for online web navigation tasks. across all test tasks, agents equipped with our meta planner consistently outperform those without it, achieving at least a 5.6% average improvement in performance. addition-ally, the meta planner is compatible with various agent training frameworks, and its meta plans can be directly inserted into task instructions. combined with these methods, our approach yields even greater performance gains, demonstrating effectiveness in a larger application scope. further analysis reveals that our generated meta plans significantly increase the agent's average reward per action, thereby improving task completion efficiency. in summary, our contributions are as follows: • we introduce the mpo, which leverages meta plan optimization to improve the performance of llm agents. this progress provides an innovative approach to explicitly enhance agents' planning capabilities while maintaining compatibility with previous agent training frameworks. ## conclusion in this paper, we introduce mpo, a novel framework for enhancing the performance of llm-based agents. mpo incorporates abstract, high-level guidance via meta plans, offering an innovative solution to explicitly improve the agent's planning capabilities. by utilizing feedback from the agent's task execution, mpo enables continuous enhancement of the meta plan quality. extensive experiments on three benchmarks demonstrate that our framework consistently outperforms existing baselines and is applicable to agents across a wide range of parameter sizes. these findings highlight the potential of our approach to advance agent planning capabilities, paving the way for future developments in artificial general intelligence.