SELFGOAL : Your Language Agents Already Know How to Achieve High-level Goals (2025)

Abstract: Language agents powered by large language models (LLMs) are increasingly valuable as decision-making tools in domains such as gaming and programming. However, these agents often face challenges in achieving high-level goals without detailed instructions and in adapting to environments where feedback is delayed. In this paper, we present SELFGOAL, a novel automatic approach designed to enhance agents’ capabilities to achieve high-level goals with limited human prior and environmental feedback. The core concept of SELFGOAL involves adaptively breaking down a high-level goal into a tree structure of more practical subgoals during the interaction with environments while identifying the most useful subgoals and progressively updating this structure. Experimental results demonstrate that SELFGOAL significantly enhances the performance of language agents across various tasks, including competitive, cooperative, and deferred feedback environments.

## introduction the advancement of large language models (llms) @xcite @xcite has enabled the construction of autonomous language agents (or llm-based agents) to solve complex tasks in dynamic environments without task-specific training. in reality, these autonomous agents are often tasked with very broad, high-level goals, such as "winning the most money" or "succeeding in a competition", whose ambiguous nature and delayed reward raise great challenges for autonomous task-solving. more importantly, it is not always practical to frequently retrain models with limited generalizability to adapt to new goals and tasks @xcite @xcite . therefore, a critical question arises: how can we enable autonomous language agents to consistently achieve high-level goals without training? previous works focus on creating two types of auxiliary guidance in the instructions for language agents to achieve high-level goals in tasks: prior task decomposition and post-hoc experience summarization. the former involves decomposing the task before acting, utilizing prior knowledge from llms to break down high-level goals into more tangible subgoals related to specific actions at hand @xcite @xcite . however, this line of work does not ground these subgoals into the environment during interaction, resulting in the loss of empirical guidance. in contrast, the latter allows agents to interact directly with environments and summarize valuable experiences from history @xcite @xcite , e.g., "x contributes to y". however, the difficulty of inducing rules from experience causes the guidance to be simple and unstructured, making it difficult to prioritize or adjust strategies effectively. a natural solution to combine the best of both worlds is to dynamically decompose the task and its high-level goal during interaction with the environment. this approach requires an agent to build and use guidelines that vary in detail and aspect. a tree structure is ideal for this requirement, as it allows hierarchical organization, providing both broad overviews and detailed guidance as needed. however, this approach presents two major challenges: 1) not all nodes are relevant to the current context during task execution, which requires selecting the most suited nodes to guide current actions. for example, "watch for bargains" is a more prudent choice than "bid on the most expensive item" when budget is tight; to tackle these challenges, we propose self-goal, a self-adaptive framework for a language agent to utilize both prior knowledge and environmental feedback to achieve high-level goals. the main idea is to build a tree of textual subgoals, where agents choose appropriate ones as the guidelines to the prompt based on the situation. specifically, as shown in figure 1 , selfgoal is featured with three main modules to operate a goaltree, which is constructed, utilized and updated during task execution: 1) search module is prompted to select the top-k most suited nodes of goals based on the provided current state and existing nodes in goaltree, which utilizes the prior knowledge of llms; 2) act module takes as input the selected subgoals as guidelines, and prompts llms for actions for the current state; 3) decomposition module breaks down selected goal nodes into a list of more concrete subgoals as subsequent leaves, ensuring an adaptive self-growth of goaltree. note that we filter out the redundant nodes during decomposition based on the textual similarity between new ones and the existing nodes of goals. extensive experiments in various competition and collaboration scenarios show that selfgoal pro-vides precise guidance for high-level goals and adapts to diverse environments, significantly improving language agent performance. in summary, our contributions in this paper are as follows: learning from feedback recently, llms have become a promising tool for building goal-directed language agents @xcite . with textual input that includes the world state, task, and interaction history, language agents are to decide the next action to achieve a goal @xcite . several studies have explored enhancing the reasoning and planning abilities of language agents through feedback from environments. for example, reflexion @xcite enables an agent to reflect on its failures and devise a new plan that accounts for previous mistakes. similarly, voyager @xcite operates in minecraft, developing a code-based skill library from detailed feedback on its failures. recent works @xcite analyze both failures and successes attempts, summarizing a memory of causal abstractions. however, learnings directly from feedback are often too general and not systematic, making it difficult to prioritize strategies effectively. ## conclusion in this paper, we introduce selfgoal, an agent framework that enhances the capabilities of llms for achieving high-level goals across various dynamic tasks and environments. we demonstrate that selfgoal significantly improves agent performance by dynamically generating and refining a hierarchical goaltree of contextual subgoals based on interactions with the environments. experiments show that this method is effective in both competitive and cooperative scenarios, outperforming baseline approaches. moreover, goaltree can be continually updated as agents with self-goal further engage with the environments, enabling them to navigate complex environments with greater precision and adaptability.