Language Models Can Infer Action Semantics for Symbolic Planners from Environment Feedback (2025)

Abstract: Symbolic planners can discover a sequence of actions from initial to goal states given expert-defined, domain-specific logical action semantics. Large Language Models (LLMs) can directly generate such sequences, but limitations in reasoning and state-tracking often result in plans that are insufficient or unexecutable. We propose Predicting Semantics of Actions with Language Models (PSALM), which automatically learns action semantics by leveraging the strengths of both symbolic planners and LLMs. PSALM repeatedly proposes and executes plans, using the LLM to partially generate plans and to infer domain-specific action semantics based on execution outcomes. PSALM maintains a belief over possible action semantics that is iteratively updated until a goal state is reached. Experiments on 7 environments show that when learning just from one goal, PSALM boosts plan success rate from 36.4% (on Claude-3.5) to 100%, and explores the environment more efficiently than prior work to infer ground truth domain action semantics.

## introduction symbolic planning requires extensive domain knowledge to produce a sequence of actions that achieve a specified goal. domain knowledge comprises expert-annotated action semantics that govern the dynamics of the environment. for example, traditional symbolic methods, like planning domain description language (pddl; @xcite solvers, take action semantics annotated in a domain file as input (figure 2 ). these symbolic solvers systematically search the state space based on actions that can be executed as per these semantics and return a sequence expected to achieve specified goal conditions, if possible. however, a human expert must exhaustively define the action semantics of domain to enable symbolic planning. we propose a novel domain induction task in which an agent must infer the action semantics of an environment without manual annotation or error correction. in this setting, domain information like object properties and action functions headers are given to an agent in natural language. the agent is then asked to infer the action semantics, that is, symbolic pre-and post-conditions, through interacting with the environment and learning from resulting feedback (figure 1 ). our proposed setting is motivated by the longer-term goal of building real-world robots that can explore a new environment (e.g., an apartment) and learn to perform new tasks in the environment (e.g., tidying) by building a symbolic, interpretable world representation. we draw inspiration from prior work that prompts large language models (llms) to perform robotics planning tasks. these methods rely on llms' ability to infer world knowledge from natural language input for generating unseen plans, rather than leveraging expert domain knowledge. for instance, progprompt @xcite generates program-like high-level plans from llms. llms struggle with long-horizon planning and often generate plans that include invalid actions, such as filling a cup before the lid is removed. instead of using the llm directly as a planner, we propose predicting semantics of actions with language models (psalm), a novel method that combines language models with symbolic solvers to iteratively explore the environment and predict the action semantics based on environment feedback. psalm maintains a probabilistic memory of learned action semantics, which iteratively improves by interacting with the environment. we use llms to sample possible incorrect or incomplete candidate action trajectories conditioned on initial proposed action sequences from a symbolic planner, then infer action semantics based on the result of executing those trajectories. psalm leverages llms' strong commonsense reasoning abilities, as well as their ability to generate syntactically valid formal semantics, while using a symbolic solver to search for ways to achieve the final goal state based on our current belief about the action semantics. we demonstrate the effectiveness and efficiency of psalm for domain induction in 7 symbolic reasoning environments. psalm can achieve 100% success rate with the fast-downward @xcite solver, while the best llms, @xcite and claude-3.5-sonnet (anthropic, 2024) perform less than 40% on average. additionally, psalm consistently induces correct domain files, and does so with substantially fewer total execution steps and environment resets than other approaches. the integration of llms and symbolic solvers is a promising avenue for domain induction in robotics and general reasoning agent workflows. ## background and related works we first introduce symbolic planning, and then compare psalm's domain induction setup with previous llm or llm-modulo planning methods, as listed in ## problem formulation symbolic planning in pddl requires a domain file characterizing the environment. human experts must carefully annotate the action semantics to enable the symbolic solver to find correct solutions. we propose a novel domain induction task, where ai agents must find the action semantics for a new domain without human annotation or correction. in the domain induction task, the agent knows d \ @xmath0, including the predicates and the action names, but not the action semantics. in addition, the agent has access to one problem file ⟨o, s i , s g ⟩. the goal of the agent is to learn the correct action semantics φ a for each action a. during the learning process, the agent interacts with the environment initialized to state s i with an openloop execution of a planned trajectory τ = a 1:t , after which the environment is reset back to s i . af- derived info provided info 4. sample preconditions and postconditions 1. sample trajectories 2. execute actions 3. generate action semantics domain info -domain desp -type -predicate -action desp problem info -task desp -problem file solver info -candidate trajs memory info -failed trajs -sampled dynamics model trajectory sampler environment symbolic planner action semantic generator sampled domain file trajectory valid actions state transitions error msg for no yes learned action semantics put-down preconditions: [ (holding ?ob): 1.0 (not (on-table ?ob)): 0.3 ] put-down postconditions: [ (arm-empty): 1.0 (on-table ?ob): 0.8 (clear ?ob): 0.6 ] unstack preconditions: [ … solution environment success yes find solution! no update memory pre-and post-conditions for each action solver info domain info problem info memory info domain info problem info memory info (unstack b3 b2) (put-down xqb2) (unstack b2 b5) (put-down b3) (stack b5 b2) … time step 1: (unstack b3 b2) adding (holding b3) deleting (on b3 b2) time step 2: (put-down b2) plan failed: unsatisfied precondition plan repair: (set (holding b2) to true) preconditions of "put-down ?ob": (holding ?ob), (not (on-table ?ob)) postconditions of "put-down ?ob": (on-table ?ob), (clear ?ob), (arm-empty) … put-down preconditions: (holding ?ob) put-down postconditions: (on-table ?ob) (arm-empty) unstack preconditions: … figure 3: the pipeline of psalm in four steps: (1) sample trajectories from a trajectory sampler; (2) execute the trajectories in the environment to get feedbacks (3) generate action semantics for each action with environment feedback, and update the memory based on the prediction; (4) sample action semantics from the memory to construct the domain file for the symbolic solver to check the success. ## proposed approach: psalm we propose the predicting semantics of actions with language models (psalm) framework, which leverages llm commonsense, formal planning, and environment feedback. ## experiments we show that psalm can recover 100% the action semantics in 7 symbolic domains, resulting in 100% planning success rate as well. the critical advantage the llm component of psalm is to greatly reduce the cost, i.e., the number of resets and the number of executed steps, for achieving the iterative recovery of these domain semantics. ## conclusion in conclusion, we propose a novel domain induction problem in pddl, where agents automatically infer the action semantics for a new domain without human annotation. we introduce a simple but strong framework psalm, which combines the commonsense reasoning ability of large language models with the precision of symbolic solvers for domain induction. to update the action semantics in a memory, psalm uses llms as agents for trajectory sampling and action semantics prediction. we demonstrate the effectiveness and efficiency of leveraging llms with environment feedback for domain induction over 7 domains.