Venue: NAACL 2025

# Language Models Can Infer Action Semantics

for Symbolic Planners from Environment Feedback

 Wang Bill Zhu   Ishika Singh    Robin Jia     Jesse Thomason

Department of Computer Science

University of Southern California

{wangzhu@usc.edu, ishikasi@usc.edu, robinjia@usc.edu, jessetho@usc.edu}

###### Abstract

Symbolic planners can discover a sequence of actions from initial to goal states given expert-defined, domain-specific logical action semantics. Large Language Models (LLMs) can directly generate such sequences, but limitations in reasoning and state-tracking often result in plans that are insufficient or unexectable. We propose Predicting Semantics of Actions with Language Models (PSALM), which automatically learns action semantics by leveraging the strengths of both symbolic planners and LLMs. PSALM repeatedly proposes and executes plans, using the LLM to partially generate plans and to infer domain-specific action semantics based on execution outcomes. PSALM maintains a belief over possible action semantics that is iteratively updated until a goal state is reached. Experiments on 7 environments show that when learning just from one goal, PSALM boosts plan success rate from 36.4% (on Claude-3.5) to 100%, and explores the environment more efficiently than prior work to infer ground truth domain action semantics.

## 1 Introduction

Symbolic planning requires extensive domain knowledge to produce a sequence of actions that achieve a specified goal. Domain knowledge comprises expert-annotated action semantics that govern the dynamics of the environment. For example, traditional symbolic methods, like Planning Domain Description Language (PDDL; Ghallab et al., 1998) solvers, take action semantics annotated in a domain file as input (Figure 2). These symbolic solvers systematically search the state space based on actions that can be executed as per these semantics and return a sequence expected to achieve specified goal conditions, if possible. However, a human expert must exhaustively define the action semantics of domain to enable symbolic planning.

We propose a novel domain induction task in which an agent must infer the action semantics of an environment without manual annotation or error correction. In this setting, domain information like object properties and action functions headers are given to an agent in natural language. The agent is then asked to infer the action semantics, that is, symbolic pre- and post-conditions, through interacting with the environment and learning from resulting feedback (Figure 1). Our proposed setting is motivated by the longer-term goal of building real-world robots that can explore a new environment (_e.g._, an apartment) and learn to perform new tasks in the environment (_e.g._, tidying) by building a symbolic, interpretable world representation.

We draw inspiration from prior work that prompts Large Language Models (LLMs) to perform robotics planning tasks. These methods rely on LLMs' ability to infer world knowledge from natural language input for generating unseen plans, rather than leveraging expert domain knowledge. For instance, ProgPromptSingh et al. (2023) generates program-like high-level plans from LLMs.

LLMs struggle with long-horizon planning and often generate plans that include invalid actions,

Figure 1: LLMs can propose plans and generate action semantics, but struggle with state tracking. Symbolic planners leverage specialized search algorithms, but require predefined action semantics for the environment. PSALM integrates the strengths of both.



such as filling a cup before the lid is removed. Instead of using the LLM directly as a planner, we propose Predicting Semantics of Actions with Language Models (PSALM), a novel method that combines language models with symbolic solvers to _iteratively explore the environment and predict the action semantics based on environment feedback_. PSALM maintains a probabilistic memory of learned action semantics, which iteratively improves by interacting with the environment. We use LLMs to sample possible incorrect or incomplete candidate action trajectories conditioned on initial proposed action sequences from a symbolic planner, then infer action semantics based on the result of executing those trajectories. PSALM leverages LLMs' strong commonsense reasoning abilities, as well as their ability to generate syntactically valid formal semantics, while using a symbolic solver to search for ways to achieve the final goal state based on our current belief about the action semantics.

We demonstrate the effectiveness and efficiency of PSALM for domain induction in 7 symbolic reasoning environments. PSALM can achieve 100% success rate with the FastDownwardHelmert (2006) solver, while the best LLMs, O1-preview OpenAI (2024) and Claude-3.5-Sonnet Anthropic (2024) perform less than 40% on average. Additionally, PSALM consistently induces correct domain files, and does so with substantially fewer total execution steps and environment resets than other approaches. The integration of LLMs and symbolic solvers is a promising avenue for domain induction in robotics and general reasoning agent workflows.

## 2 Background and Related Works

We first introduce symbolic planning, and then compare PSALM's domain induction setup with previous LLM or LLM-Modulo planning methods, as listed in Table 1.

### Classical and Symbolic Planning

Classical planning algorithms have been widely applied in autonomous spacecrafts, military logistics, manufacturing, games, and robotics. The automated STRIPS planner was the first algorithm that operated the Shakey robot Fikes and Nilsson (1971). Classical planners require finite, deterministic, and full state information to generate guaranteed plans when a path from the initial to the goal state is possible. Some other frameworks were also shown to be useful for robot planning Carbonell et al. (1991); Nau et al. (2003). Symbolic planning languages, such as Planning Domain Description Language (PDDL; Ghallab et al. (1998)) and Answer Set Programming (ASP; Brewka et al. (2011); Lifschitz (2002)), provide a more structured and flexible way to represent problems.

We use PDDL for symbolic planning Figure 2). We define a planning problem P as a tuple \(\langle{\cal D},{\cal O},s^{i},{\cal S}^{g}\rangle\), for \({\cal D}\) the domain, \({\cal O}\) a set of objects in the domain, \(s^{i}\) the initial state, and \({\cal S}^{g}\) the goal specification, _i.e._, a set of goal states satisfying the goal conditions. A plan solution to P is a \(T\)-step sequence of actions \(a_{1..T}\), which once executed at \(s^{i}\) would lead to a state in \({\cal S}^{g}\).

The PDDL domain file also defines the symbolic action set \({\cal A}\) in the environment and _predicates_ representing objects properties. Each action has a string name (_e.g._, Put-down), parameters (_e.g._, ?ob), and _semantics_. The action semantics \(\Phi_{a}\) of an action \(a\in{\cal A}\) include the _preconditions_ when \(a\) is valid to execute, and the _postconditions_ (effects) describing what changes when an \(a\) is executed. Unlike LLM+P Liu et al. (2023), which generates PDDL problem files from in-context examples, we infer action semantics in domain files.

### LLMs for Task Planning

Several works utilize LLMs to generate plans directly given initial and goal states Huang et al. (2022); Ichter et al. (2022), but such stochastic, generative approaches lose the success guarantees of symbolic planners. To improve the correctness of the plans, learned latent spaces Trivedi et al. (2021) or LLMs Singh et al. (2023); Liang et al. (2023); Hu et al. (2024) are used to generate executable programs, which introduce some symbolic structure and constraints, reaching up to 90% plan success on some simple domains, such as Grippers, but these still do not guarantee plan success Silver et al. (2023).

Recently, Kambhampati et al. (2024) proposed the LLM-Modulo network, combining LLM plan generation with symbolic planner verifiers. Using an LLM for final plan generation cannot achieve high success rate in complex domains, such as Barman and TermesZhao et al. (2023); Hazra et al. (2024). Guan et al. (2023) have compared LLMs and symbolic planners, finding the latter is consistently better over 3 domains.

In this paper, we consider planning domain induction. We use LLMs to propose partial plans 

and predict domain knowledge based on execution feedback. Once we uncover the domain action semantics, plan success from a symbolic solver is guaranteed, provided the goal is reachable.

### Domain Induction

The domain induction problem has been studied under different setups (Appendix Table 4). Arora et al. (2018) introduced several setups, including predicting pre- and post-conditions from valid plans or reusable plan fragments. Wong et al. (2023) and Chen et al. (2024) filled in missing pre- and post-conditions in the domain file given other action semantics in the same or similar domain. Guan et al. (2023) generated domain predicates and action semantics using LLMs given detailed text action descriptions. Because the generated predicates were not aligned with the problem file, human feedback was required for error correction and evaluation.

Closest to our paper, Oswald et al. (2024) extended Guan et al. (2023)'s work by providing predicates and enabling automatic evaluation. They performed a single-round action semantics generation from LLMs and evaluated the accuracy of the generation. Using environment feedback, PSALM effectively solves their domain induction problem, achieving 100% accuracy over 7 domains. We advocate for an additional evaluation relevant for downstream, real-world deployment: the efficiency of the environment exploration.

## 3 Problem Formulation

Symbolic planning in PDDL requires a domain file characterizing the environment. Human experts must carefully annotate the action semantics to enable the symbolic solver to find correct solutions. We propose a novel _domain induction_ task, where AI agents must find the action semantics for a new domain without human annotation or correction.

In the domain induction task, the agent knows \({\cal D}\setminus\bigcup_{a\in{\cal A}}\Phi_{a}\), including the predicates and the action names, but not the action semantics. In addition, the agent has access to one problem file \(\langle{\cal O},s^{i},{\cal S}^{g}\rangle\). The goal of the agent is to learn the correct action semantics \(\Phi_{a}\) for each action \(a\). During the learning process, the agent interacts with the environment initialized to state \(s^{i}\) with an open-loop execution of a planned trajectory \(\tau=a_{1:T}\), after which the environment is reset back to \(s^{i}\). Af

\begin{table}
\begin{tabular}{l c c c c c}  & **Goal** & \multicolumn{2}{c}{**Independent of**} & \multicolumn{2}{c}{**Leveraging**} & **Guaranteed** \\  & & Partial AS & Valid plans & Human eval & Env feedback & Plan success \\ \hline Liu et al. (2023) & Problem File & ✗ & ✓ & ✓ & ✗ & ✗ \\ Hazra et al. (2024) & Plan & ✓ & ✓ & ✓ & ✗ & ✗ \\ Silver et al. (2023) & Program & ✗ & ✓ & ✓ & ✗ & ✗ \\ \hline Arora et al. (2018) & AS & ✓ & ✗ & ✓ & ✗ & ✗ \\ Wong et al. (2023) & AS & ✗ & ✓ & ✓ & ✗ & ✗ \\ Guan et al. (2023) & AS & ✓ & ✓ & ✗ & ✗ & ✗ \\ Oswald et al. (2024) & AS & ✓ & ✓ & ✓ & ✗ & ✗ \\ \hline PSALM & AS & ✓ & ✓ & ✓ & ✓ & ✓ \\ \end{tabular}
\end{table}
Table 1: Comparing the PSALM domain induction task setup to representative related works in LLM and LLVMolino planning; here, “AS” is short for action semantics.

Figure 2: An example of symbolic planning information from the BlocksW domain, from left to right: PDDL domain file, PDDL problem file, visualization of initial and goal state for block stacking, and a potential plan.

 

ter learning action semantics, the agent should be able to leverage a symbolic solver for efficient and robust task solving in the domain to achieve a state in \({\cal S}^{g}\).

We evaluate the domain induction learning process on three measures. (1) Accuracy (Acc): \(\sum_{a\in{\cal A}}|\hat{\Phi}_{a}\cap\Phi_{a}|/\sum_{a\in{\cal A}}|\Phi_{a}|\), where \(\hat{\Phi}_{a}\) is the predicted action semantics for action \(a\). Finding a path to \({\cal S}^{g}\) is not sufficient to imply correct action semantics, but recovering ground truth action semantics is sufficient to reach \({\cal S}^{g}\) via the symbolic solver. (2) The number of resets (NR): how many times the environment returns to \(s_{i}\) because planned actions did not reach \({\cal S}^{g}\). (3) The number of executed steps (NES): how many total actions were executed during learning, including those causing failure.

## 4 Proposed Approach: PSALM

We propose the Predicting Semantics of Actions with Language Models (PSALM) framework, which leverages LLM commonsense, formal planning, and environment feedback.

### Overview

Given a goal \({\cal S}_{g}\), we first use an LLM as a _trajectory sampler_, and execute the trajectories in the environment to get feedback (Figure 3). We use the LLM again, together with a rule-based parser, as the _action semantics generator_ to predict action semantics, _i.e_., preconditions and postconditions, for each action based on that feedback. We update the memory of the learned action semantics, ten sample that memory for hypothesized action semantics used as input to the symbolic solver. Finally, the symbolic solver tries to generate a plan from the goal and hypothesized action semantics.

If the symbolic solver finds a solution, we execute the plan in the environment. If the plan reaches the goal, we finish the loop; otherwise, we will pass the result of the failed plan to the LLM to predict the action semantics again. If the symbolic solver does not find a solution, we will provide some partial candidate trajectories from the symbolic solver to the trajectory sampler as a prefix sequence of actions for the plan to be generated.

### Trajectory sampler

We prompt an LLM with domain and problem information (as described in SS2.1), memory information representing our current beliefs about action pre- and post-conditions, and partial trajectories generated by the solver. We use templates to convert

Figure 3: The pipeline of PSALM in four steps: (1) sample trajectories from a trajectory sampler; (2) execute the trajectories in the environment to get feedbacks (3) generate action semantics for each action with environment feedback, and update the memory based on the prediction; (4) sample action semantics from the memory to construct the domain file for the symbolic solver to check the success.

 

these into a natural language prompt. For example, the postconditions (on-table?ob) (arm-empty) of the action put-down are converted to natural language _The effects are (on-table?obj), (arm-empty)_. Because the symbolic solver we use, _i.e._, FastDownward, is performing a search algorithm given a time-limit of \(W\), if the solver finds any candidate trajectories, _i.e._, not a complete solution but a partial solution stopped by the timer or a dead end, we include the \(k\) longest candidate trajectories as additional input to the trajectory sampler. We filter out invalid candidate trajectories that failed in previous iterations. The trajectory sampler prompt finally specifies to pick one of the candidate trajectories and generate a trajectory starting from that. We execute \(l\) trajectories sampled in this way from the trajectory sampler. When \(l>1\), we predict action semantics for each trajectory separately (SS4.3). After execution, we update the semantic hypothesis memory (SS4.4).

Prospection.Although the LLM's prompt includes the current beliefs about action preconditions, the LLM can still generate trajectories that violate these preconditions. We add _trajectory prospection_, enabling the system to do forward prediction in an open-loop fashion before actually invoking the simulator based on its symbolic understanding of the world. For a generated trajectory \(\tau=a_{1:T}\), we check if is aligned with the current belief of the environment action semantics for \(v\) steps. For \(v\leq T\), if any action in \(a_{1:v}\) does not satisfy the preconditions in the sampled action semantics, starting from that action, we will keep randomly sampling one action until the sampled action is valid in the hypothesized action semantics, repeated to \(v\) total actions. Otherwise, if \(a_{1:v}\) are all valid, we execute the original \(a_{1:T}\).

Random sampler.We create an ablated PSALM that randomly samples \(v\) actions per trajectory from the longest candidate trajectory, if available, from the symbolic solver during each iteration. This random sampler ablation can also take advantage of the prospection module.

### Action semantics generator

We combine LLM-based and rule-based action semantics generation given environment feedback.

LLM-based generator.We prompt the LLM to generate the action semantics of each action separately. The prompt contains information about the domain, problem, memory information, and environment feedback. For a trajectory \(\tau=a_{1:T}\) failed at step \(t+1\), environment feedback for the LLM action semantics generator takes the form of valid actions \(a_{1:t}\), state transitions \(s_{j+1}-s_{j},j=1,\ldots,t\), where \(s_{1}\) is the initial state \(s^{i}\), and the environment error message string for action \(a_{t+1}\). Note that each state is a set of predicate assignments, so we can compute the difference of state transition as the difference of two sets. We assume the error message is provided by the environment. In the tested environments, when an action fails the error message specifies an unsatisfied precondition. When no action fails, the error message either states that the the goal is not reached, or the goal is reached and the PSALM inference loop exits. For each action, we prompt the LLM once to predict the preconditions and once for the postconditions.

Rule-based generator.We also write a rule-based parser for the output from the environment feedback to infer one or more missing preconditions suggested by error messages. Rule-based parser also scan state transition descriptions to derive postconditions. For each iteration, we update the action semantics hypothesis memory with both

\begin{table}
\begin{tabular}{l c c c c c c r} Model & Barman BlocksW & Floortile & Grippers & Storage & Termes & Tyroworld \\ \hline \multicolumn{8}{l}{_Direct plan generation_} \\ O1-preview & 0 & 30 & 0 & 90 & 5 & 0 & 30 \\ GPT-4-Turbo & 0 & 20 & 0 & 55 & 5 & 0 & 70 \\ Claude 3.5 Sonnet & 0 & 60 & 0 & 90 & 5 & 0 & 100 \\ \hline \multicolumn{8}{l}{_Domain induction + Symbolic planner_} \\ GPT-4-Turbo & 0 & 0 & 0 & 0 & 5 & 0 & **0** \\ Oswald et al. (2024) & 0 & 0 & 0 & 0 & 5 & 0 & **0** \\ PSALM & **100** & **100** & **100** & **100** & **100** & **100** \\ \hline \end{tabular}
\end{table}
Table 2: Plan success rate over 20 examples, comparing PSALM with LLM and domain induction baselines.

 

LLM-generated pre- and post-conditions and those from this rule-based generator. Unlike the LLM action semantics generator, the rule-based generator relies solely on the feedback at the current time step, making it less efficient.

### Memory of action semantics

We keep a memory of the hypothesized action semantics. For each action's action semantics, we store two lists of predicted statements for preconditions and postconditions. Each statement \(\phi\) is associated with a belief \(p(\phi\in\Phi_{a}|a)\), in short \(p(\phi|a)\). We use this belief as a binary probability for each statement to sample, at each iteration, the concrete action semantics input to the symbolic solver.

The first time a statement \(\phi\) is predicted as part of the action semantics for \(a\), the belief will be assigned to \(1\). Afterwards, this belief will be updated following an exponential forgetting rule. Suppose at time step \(t\), the predicted action semantics of \(a\) is \(\hat{\Phi}_{a,t}\), the belief update rule of statement \(\phi\) is

\[p_{t+1}(\phi|a)=\] \[\begin{cases}\mathbbm{1}[\phi\in\hat{\Phi}_{a,t+1}]&\text{if $ p_{t}(\phi|a)=0$}\\ \gamma_{\phi}p_{t}(\phi|a)+(1-\gamma_{\phi})\mathbbm{1}[\phi\in\hat{\Phi}_{a, t+1}]&\text{else}\end{cases}\]

where \(\gamma_{\phi}\) is the forgetting factor. If a statement \(\phi\) has only been predicted by the LLM. Once a statement \(\phi\) is predicted by rule-based generator, \(\gamma_{\phi}=1\), which does not decay. Notice that all the statements predicted in the current time step and in the memory \(\phi\in\bigcup_{t^{\prime}\in\{1..t+1\}}\hat{\Phi}_{a,t^{\prime}}\) will be updated following the rule.

## 5 Experiments

We show that PSALM can recover 100% the action semantics in 7 symbolic domains, resulting in 100% planning success rate as well. The critical advantage the LLM component of PSALM is to greatly reduce the cost, _i.e._, the number of resets and the number of executed steps, for achieving the iterative recovery of these domain semantics.

### Experimental setups

We experiment on 7 symbolic domains from International Planning Competitions Seipp et al. (2022); each defines 20 tasks that vary in number of environment objects and optimal plan length. **(1)****Barman**: The robot is a bartender with 2 hands preparing cocktails for a customer's order, using the specified ingredients and appropriate tools; **(2)****BlocksW**: The robot reorganizes a collection of block piles arranged on a table, into a specified configuration while adhering to the simple physics principles; **(3)****Floortle**: A set of robots painting color patterns on floor tiles, allowed to move around but not to step on painted tiles; **(4)****Grippers**: A set of robots with 2 grippers each are given a task to move objects among different rooms; **(5)****Storage**: The robot lifts and drops crates initially stored in different areas, into a depot, using a given set of hoists; **(6)****Termes**: The robot constructs complex structures by transporting and positioning blocks, as well as using them as a means to move adjacent blocks; **(7)****Tyreworld**: The robot is assigned with changing flat tries, which involves tasks such as removing flat tires, inflating the intact tires, tightening nuts, and returning tools to the boot.

In each domain, we choose the first task that requires using all actions to reach the goal, learn action semantics from that task, then test on all 20 tasks. All experiments use the FastDownwardHelmert (2006) planner, with a search time limit of \(W=30\) seconds during the PSALM loop, and unlimited search time during testing (Table 2). We use VAL1 as simulation environment for plan validation, calculating state condition changes, and obtaining error messages. We set the maximum number of PSALM induction iterations to 1k for pure random baselines and to 100 for any method involving LLMs. We report the average over 3 runs for all the methods.

Footnote 1: [https://github.com/KCL-Planning/VAL](https://github.com/KCL-Planning/VAL)

For the main results, we use GPT-4-Turbo OpnemAI (2023) as the language model agent, with tem

\begin{table}
\begin{tabular}{l r r r r r r} Model & Barman BlocksW Floortile Grippers & Storage Termes & Tyreworld \\ \hline GPT-4-Turbo & 53 & 61 & 58 & 70 & 45 & 41 & 67 \\ Oswald et al.Oswald et al. (2024) & 65 & 67 & 58 & 75 & 58 & 59 & 80 \\ PSALM & **100** & **100** & **100** & **100** & **100** & **100** & **100** \\ \hline \end{tabular}
\end{table}
Table 3: Action semantics accuracy over 20 examples, comparing PSALM with LLM to baselines.

 

perature 0 and one-shot prompting following Liu et al. (2023). We detail prompts in Appendix C. We use \(v=10\) prospection steps, \(l=1\) sampled trajectory, \(k=3\) candidate trajectories, \(g=5\) failed trajectories per iteration, \(\gamma_{\phi}=0.8\) memory forgetting factor. We perform ablation studies over these hyperparameters in the analysis. More experimental details are in Appendix B.

### Comparison with baselines

As shown in Table 2, PSALM achieve 100% planning success rate, outperforming multiple LLM direct plan generation baselines. Other domain induction baselines using LLMs, such as Guan et al. (2023), are unable induce correct action semantics because they do not leverage environment feedback. Direct prompting for the plan of the task, even with powerful LLM models, O1-preview and Claude-3.5 Sonnet, cannot solve any task in the complex domains such as Barman or Termes.

Table 3 shows that without environment feedback, GPT-4 and Guan et al. (2023) can predict part of the action semantics of the environment, but cannot recover the full action semantics correctly. Given only partially correct action semantics, the solver will either fail to find a plan, or generate a plan that fails when executed. We provide the prompt templates for plan generation and domain induction as in Appendix C.

### Ablations on using LLM and prospection

Figure 4 compares PSALM with multiple baselines over 7 domains, including no rule-based action semantics generator (w/o rule-ASG), no LLM action semantics generator (w/o LLM-ASG), no prospection (w/o prospect.) and use random trajectory sampler (w/o LLM-TS). Each achieves 100% Acc, so we focus on number of executed steps (NES).

LLM as a sampler greatly reduces the execution steps.We find that the LLM trajectory sampler reduces the execution steps over all domains. The random sampler, even with prospection, lacks the commonsense reasoning knowledge for choosing trajectories likely to be solutions to the problem.

LLM and rule-based action semantics predictions have complementary benefits.Comparing the purple bars (w/o rule-ASG) and the magenta bars (w/o LLM-ASG), rule-based action semantics generator is more important in Flortile, Storage and Tyreworld, while LLM action semantics generator is more important in the other four domains. With the commonsense prior knowledge provided by the LLM, and the exactness of the information from the rule-based parser, combining both predictions is always a better solution.

Prospection induces redundant actions sometimes, but is necessary overall.For certain do

Figure 4: We compare PSALM with multiple variations over 7 domains. We report on NES and the results suggest (1) LLM as a trajectory sampler greatly reduces the execution steps; (2) LLM and rule-based action semantics generators have complementary benefits; and (3) Prospection to reject trajectories based on current action semantics hypotheses is helpful overall. TS is short for trajectory sampler and ASG is short for action semantics generator.

 

mains with a small number of actions \(|{\cal A}|\), such as Grippers (\(|{\cal A}|=3\)) and Termes (\(|{\cal A}|=7\)), a large number of prospection steps slows search. However, prospection is overall helpful. The w/o prospect. bar shows around 1.4 times execution steps on average PSALM. Especially, in the Barman domain, prospection reduces more than a half of the execution steps.

### Analysis

PSALM can work on less powerful and public LLMs.Figure5 left shows PSALM can work with 100% plan success rate on GPT-3.5 and Mistral-7B, though with more resets (NR) and number of execution steps (NES). Prospection reduces the gap between the different LLMs.

LLM prior enables random baselines.We experiment with using the LLM to predict the action semantics \(\hat{\Phi}_{0,a}\) for each action \(a\), before having any sampled trajectory, and then start the iteration with the memory of the dynamics model initialized from \(\hat{\Phi}_{0,a}\). This prediction is solely based on the name of the action and its parameter. Figure5 middle shows on the Termes domain the LLM prior can enable the random baseline without prospection to work within 1k resets, and greatly reduce NR and NES for the random baseline with prospection. However, on more complex domains such as Barman, the LLM prior does not enable the random baselines, _i.e_., failing to find a solution within 1k resets, with and without prospection. The finding implies we can perform domain induction on some easier domains with limited access, or even one call, of language models. On the other hand, this naive LLM prior is harmful to PSALM, because it inserts noisy predictions of action semantics which requires additional resets to erase.

Error messages are crucial for domain induction.Error messages are obtainable from simulation environments, or in real world deployment if human operators are available to describe failure. We study whether PSALM can succeed without receiving error messages about failed actions. Note that without the error message, the action semantics generator can only be LLM-based and cannot utilize rule-based inference. Figure5 right shows that on slightly complex domains like Termes, PSALM cannot predict the correct dynamics model, _i.e_., the accuracy is not 100. PSALM works on very simple domains like Grippers, but it requires 7x more NES to learn. Moreover, prospection is essential when we have no error message.

### Sensitivity tests

We perform ablation studies on the sensitivity of prospection steps, number of sampled trajectories, number of candidate trajectories, and number of failed trajectories per run, to show their influence on the PSALM framework.

Figure 5: Additional analysis for PSALM. (Left) We vary the type of LLM and show that PSALM works with GPT-3.5 and Mistral-7B on the Termes domain. (Middle) Using the LLM prior before trajectory sampling (darker bars) enables the random baselines to work better compared to not having the prior (lighter bars), though it can adversely affect the full PSALM method. (Right) Experiments where we remove the error message from input to the LLM action semantics generator. Without error messages, PSALM works only on easy domains. For the experiments that fail to find a solution of the problem, we show the action semantics accuracy on top of the bar.

 



**Few prospection steps is enough for PSALM.** We vary prospection steps \(v\) from \(0,1,5,10\) in Figure 6a. Notice the random baseline does not find a solution when \(v=0,1,5\), and reaches an Acc of 78.5. We conclude the number of prospection steps matters more when the trajectory sampler is random, while a small number of prospection is enough for language model trajectory sampler.

One sampled trajectory per run is enough.We vary the number of sampled trajectories \(l\) from \(1,3,5\) in Figure 6b. For both LLM and random samplers, We see the total number of steps grows linearly with number of sampled trajectory, which means LLMs are hard to learn more information from just sampling more each run, and one sampled trajectory per run is enough.

More candidate trajectories help on prospection.We vary the number of candidate trajectories \(k\) from \(0,1,3\) in the LLM prompt. The results in Figure 6c suggest including candidate trajectory in the prompt is beneficial, while one candidate trajectory is enough for pure LLM prediction, more candidate trajectories help LLM with prospection. On the other hand, as the candidate trajectories can be very long for certain domain like Barman, and thus result in a very long prompt, we do not experiment with more than 3 candidate trajectories in the prompt.

Choose the number of failed trajectories with care.We vary the number of failed trajectories \(g\) from \(0,1,5\) in the LLM prompt. To select \(g\) failed trajectories, we first filter trajectories with no less than 3 steps, and then sample \(g\) trajectories from the filtered ones for the LLM prompt to avoid generating those trajectories as prefix. The results in Figure 6d suggest that multiple failed trajectories help LLM action semantics prediction, but only one failed trajectory could be harmful. We hypothesize the one failed trajectory in the input might distract the model from following the candidate trajectories.

## 6 Conclusion

In conclusion, we propose a novel domain induction problem in PDDL, where agents automatically infer the action semantics for a new domain without human annotation. We introduce a simple but strong framework PSALM, which combines the commonsense reasoning ability of large language models with the precision of symbolic solvers for domain induction. To update the action semantics in a memory, PSALM uses LLMs as agents for trajectory sampling and action semantics prediction. We demonstrate the effectiveness and efficiency of leveraging LLMs with environment feedback for domain induction over 7 domains.

### Limitations

Though PSALM shows to be relatively robust across multiple LLMs, we do not claim that PSALM can work with _any LLM_. Additionally, the error message, which might be hard to get from some real-world environment, is crucial in the PSALM framework given current LLM reasoning abilities. Future works may explore methods without the error message as the input. To apply the domain induction setup to the real world, the environment predicates and object types have to be annotated or derived from the environment. The

Figure 6: Ablation studies for PSALM. (a). Varying the number of prospection steps \(v\). Few prospection steps is enough for PSALM. (b). Varying the number of sampled trajectories \(l\). One sampled trajectory per run is enough. (c). Varying the number of candidate trajectories \(k\) passed from the symbolic solver to the LLM trajectory sampler. More candidate trajectories help when prospection is used. (d). Varying the number of failed trajectories \(g\) shown to the LLM trajectory sampler. A certain amount of failed trajectories is required for the LLM.



domain induction problem without these annotations is more challenging, but expert definitions of these domain components represent substantially less effort than fully specifying action pre- and post-conditions. We will leave this extension for future studies.

## Acknowledgments and Disclosure of Funding

WZ and RJ were supported in part by a grant from Open Philanthropy. RJ was supported in part by the National Science Foundation under Grant No. IIS-2403436. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. IS and JT were supported in part by a grant from the Army Research Lab (ARL) Army AI Innovations Institute (A2I2), award number W911NF-23-2-0010.

## References

* Amiri et al. (2019) Saeid Amiri, Sujay Bajracharya, Cihangir Goktolgal, Jesse Thomason, and Shiqi Zhang. 2019. Augmenting knowledge through statistical, goal-oriented human-robot dialog. In _2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_.
* Anthropic (2024) Anthropic. 2024. Claude 3.5 sonnet. [https://www.anthropic.com/news/claude-3-5-sonnet](https://www.anthropic.com/news/claude-3-5-sonnet).
* Arora et al. (2018) Ankuj Arora, Humbert Fiorino, Damien Pellier, Marc M ' Etivier, and Sylvie Pesty. 2018. A Review of Learning Planning Action Models. _Knowledge Engineering Review_, 33.
* Brewka et al. (2011) Gerhard Brewka, Thomas Eiter, and Miroslaw Truszczynski. 2011. Answer set programming at a glance. _Commun. ACM_.
* Carbonell et al. (1991) Jaime Carbonell, Oren Etzioni, Yolanda Gil, Robert Joseph, Craig Knoblock, Steve Minton, and Manuela Veloso. 1991. Prodigy: An integrated architecture for planning and learning. _SIGART Bull._
* Chen et al. (2024) Guanqi Chen, Lei Yang, Ruixing Jia, Zhe Hu, Yizhou Chen, Wei Zhang, Wenping Wang, and Jia Pan. 2024. Language-augmented symbolic planner for open-world task planning. In _Robotics: Science and Systems Conference (RSS)_.
* Fikes and Nils (1971) Richard E. Fikes and Nils J. Nilsson. 1971. Strips: A new approach to the application of theorem proving to problem solving. _Artificial Intelligence_.
* the planning domain definition language. _Technical Report, Tech. Rep._
* Guan et al. (2023) Lin Guan, Karthik Valmeekam, Sarath Sreedharan, and Subbarao Kambhampati. 2023. Leveraging pretrained large language models to construct and utilize world models for model-based task planning. In _Thirty-seventh Conference on Neural Information Processing Systems_.
* Han et al. (2024) Muzhi Han, Yifeng Zhu, Song-Chun Zhu, Ying Nian Wu, and Yuke Zhu. 2024. Interpret: Interactive predicate learning from language feedback for generalizable task planning. In _Robotics: Science and Systems (RSS)_.
* Hazra et al. (2024) Rishi Hazra, Pedro Zuidberg Dos Martirres, and Luc De Raedt. 2024. Saycampay: Heuristic planning with large language models using learnable domain knowledge. In _Proceedings of the AAAI Conference on Artificial Intelligence_, volume 38, pages 20123-20133.
* Helmert (2006) Malte Helmert. 2006. The fast downward planning system. _J. Artif. Int. Res._, 26(1):191-246.
* Hu et al. (2024) Zichao Hu, Francesca Lucchetti, Claire Schlesinger, Yash Saxena, Anders Freeman, Sadanand Modak, Arjun Guha, and Joydeep Biswas. 2024. Delopying and evaluating llms to program service mobile robots. _IEEE Robotics and Automation Letters_, 9(3):2853-2860.
* Huang et al. (2022) Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. 2022. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In _International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA_, Proceedings of Machine Learning Research.
* Ichter et al. (1991) Brian Ichter, Anthony Brohan, Yevgen Chebotar, Chelsea Finn, Karol Hausman, Alexander Herzog, Daniel Ho, Julian Ibarz, Alex Irpan, Eric Jang, Ryan Julian, Dmitry Kalashnikov, Sergey Levine, Yao Lu, Carolina Parada, Konishka Rao, Pierre Sermanet, Alexander T Toshev, Vincent Vanhoucke, Fei Xia, Ted Xiao, Peng Xu, Mengyuan Yan, Noah Brown, Michael Ahn, Omar Cortes, Nicolas Sievers, Clayton Tan, Sichuan Xu, Diego Reyes, Jarek Rettinghouse, Jornell Quaimbao, Peter Pastor, Linda Luu, Kuang-Huei Lee, Yuheng Kuang, Sally Jesmonth, Kyle Jeffrey, Rosario Jauregui Ruano, Jasmine Hsu, Keerthan Gopalakrishnan, Byron David, Andy Zeng, and Chuyuan Kelly Fu. 2022. Do as i can, not as i say: Grounding language in robotic affordances. In _6th Annual Conference on Robot Learning_.
* Kambhampati et al. (2024) Subbarao Kambhampati, Karthik Valmcekm, Lin Guan, Mudit Verma, Kaya Stechly, Siddhant Bhambri, Lucas Paul Saldyt, and Anil B Murthy. 2024. Position: LLMs can't plan, but can help planning in LLM-modulo frameworks. In _Forty-first International Conference on Machine Learning_ 

Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. 2023. Code as policies: Language model programs for embodied control. In _2023 IEEE International Conference on Robotics and Automation (ICRA)_.
* Lifschitz [2002] Vladimir Lifschitz. 2002. Answer set programming and plan generation. _Artificial Intelligence_.
* Lin et al. [2024] Xinrui Lin, Yangfan Wu, Huanyu Yang, Yu Zhang, Yanyong Zhang, and Jianmin Ji. 2024. CLMASP: Coupling large language models with answer set programming for robotic task planning. _ArXiv_.
* Liu et al. [2023] Bo Liu, Yuqian Jiang, Xiaohan Zhang, Qiang Liu, Shiqi Zhang, Joydeep Biswas, and Peter Stone. 2023. LLM+P: Empowering large language models with optimal planning proficiency. _ArXiv preprint_.
* Mavrogiannis et al. [2023] Angelos Mavrogiannis, Christoforos Mavrogiannis, and Yiannis Aloimonos. 2023. Cook2tl: Translating cooking recipes to ltl formulae using large language models. _arXiv preprint arXiv:2310.00163_.
* Nau et al. [2003] Dana Nau, Tsz-Chiu Au, Okhtay Ilghami, Ugur Kuter, J William Murdock, Dan Wu, and Fusun Yaman. 2003. Shop2: An htn planning system. _J. Artif. Intell. Res. (JAIR)_.
* Olaussson et al. [2023] Theo Olaussson, Alex Gu, Ben Lipkin, Cedegao Zhang, Armando Solar-Lezama, Joshua Tenenbaum, and Roger Levy. 2023. LINC: A neurosymbolic approach for logical reasoning by combining language models with first-order logic provers. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_, pages 5153-5176, Singapore. Association for Computational Linguistics.
* OpenAI [2023] OpenAI. 2023. Gpt-4 technical report. _ArXiv preprint_, abs/2303.08774.
* OpenAI [2024] OpenAI. 2024. Introducing openai o1-preview. [https://openai.com/index/introducing-openai-o1-preview](https://openai.com/index/introducing-openai-o1-preview).
* Oswald et al. [2024] James Oswald, Kavitha Srinivas, Harsha Kokel, Junkyu Lee, Michael Katz, and Shirin Sohrabi. 2024. Large language models as planning domain generators. In _Proceedings of the International Conference on Automated Planning and Scheduling_, volume 34, pages 423-431.
* Perera et al. [2015] Vittorio Perera, Robin Soetens, Thomas Kollar, Mehdi Samadi, Yichao Sun, Daniele Nardi, Rene van de Molengraft, and Manuela Veloso. 2015. Learning task knowledge from dialog and web access. _Robotics_.
* Prasad et al. [2024] Archiki Prasad, Alexander Koller, Mareike Hartmann, Peter Clark, Ashish Sabharwal, Mohit Bansal, and Tushar Khot. 2024. ADaPT: As-needed decomposition and planning with language models. In _Findings of the Association for Computational Linguistics: NAACL 2024_, pages 4226-4252, Mexico City, Mexico. Association for Computational Linguistics.
* Sarch et al. [2023] Gabriel Sarch, Yue Wu, Michael Tarr, and Katerina Fragkiadaki. 2023. Open-ended instructable embodied agents with memory-augmented large language models. In _Findings of the Association for Computational Linguistics: EMNLP 2023_.
* Seipp et al. [2022] Jendrik Seipp, Alvaro Torralba, and Jorg Hoffmann. 2022. PDDL generators.
* Silver et al. [2023] Tom Silver, Boham Dan, Kavitha Srinivas, Joshua B. Tenenbaum, Leslie Pack Kaelbling, and Michael Katz. 2023. Generalized planning in pddl domains with pretrained large language models. In _AAAI Conference on Artificial Intelligence_.
* Singh et al. [2023] Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and animesh Garg. 2023. Progrppropt: Generating situated robot task plans using large language models. In _2023 IEEE International Conference on Robotics and Automation (ICRA)_.
* Smirnov et al. [2024] Pavel Smirnov, Frank Joublin, Antonello Ceravola, and Michael Gienger. 2024. Generating consistent pddl domains with large language models. _ArXiv preprint_.
* Trivedi et al. [2021] Dweep Trivedi, Jesse Zhang, Shao-Hua Sun, and Joseph J. Lim. 2021. Learning to synthesize programs as interpretable and generalizable policies. In _Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual_.
* a critical investigation. In _Thirty-seventh Conference on Neural Information Processing Systems_.
* Wong et al. [2023] Li Shang Wong, Jiayuan Mao, Pratyusha Sharma, Zachary S. Siegel, Jiahai Feng, Noa Korneev, Joshua B. Tenenbaum, and Jacob Andreas. 2023. Learning adaptive planning representations with natural language guidance. In _The International Conference on Learning Representations (ICLR)_.
* Zhang et al. [2024] Li Zhang, Peter Jansen, Tianyi Zhang, Peter Clark, Chris Callison-Burch, and Niket Tandon. 2024. PDDLEGO: Iterative planning in textual environments. In _Proceedings of the 13th Joint Conference on Lexical and Computational Semantics (*SEM 2024)_, pages 212-221, Mexico City, Mexico. Association for Computational Linguistics.
* Zhao et al. [2023] Zirui Zhao, Wee Sun Lee, and David Hsu. 2023. Large language models as commonsense knowledge for large-scale task planning. In _Advances in Neural Information Processing Systems_, volume 36, pages 31967-31987. Curran Associates, Inc.
* Zhu et al. [2023] Wang Zhu, Jesse Thomason, and Robin Jia. 2023. Chain-of-questions training with latent answers for robust multistep question answering. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_, Singapore. Association for Computational Linguistics 

Max Zuo, Francisco Piedrahita Velez, Xiaochen Li, Michael L. Littman, and Stephen H. Bach. 2024. Planetarium: A rigorous benchmark for translating text to structured planning languages. _arXiv_.

 

More Related Works

As shown in Table 1, the previous works have not leveraged environment feedback and guaranteed plan success as PSALM. To discuss the related works comprehensively in using LLM in planning and PDDL generation over many setups, we extend the related work as in Table 4.

### LLM as an Idea Generator in Planning

LLMs are widely used as idea generators, not only in planning, but also in other reasoning areas, such as first-order logic Olaussson et al. (2023), multistep reasoning Zhu et al. (2023), decomposition Prasad et al. (2024) and formal language Mavrogiannis et al. (2023).

The major difference between LLM idea generators in planning is whether LLMs generate ideas on the plan itself or on the action semantics. Previous works explore different ways of using LLMs to generate the plan itself, such as Monte Carlo tree search over possible plans Hazra et al. (2024), or applying other verifier or postprocessor after the plan generation Lin et al. (2024). These approaches are better than the direct prompting using LLM, while they still cannot guarantee plan success.

Alternatively, using LLMs to generate action semantics followed by a symbolic planner often results in an all-or-nothing issue during plan generation. Either the action semantics are fully captured, leading to a successful plan, or the planner fails to produce a correct plan for the domain at all. Most previous works focus on error analysis. For instance, Smirnov et al. (2024) attempts to generate syntactically correct but not semantically guaranteed PDDL domain using LLMs, by exhaustively listing syntax errors in the prompt to correct the generated domains. Oswald et al. (2024) utilizes LLM to generate PDDL actions given detailed text action descriptions and analysis of the errors. Other works like InterPreT Han et al. (2024) leverages language feedback from human during embodied interaction for domain predicates and action semantics prediction. They explore complex visual environments and show how to accomplish certain human specified goal with robots, such as "stack red block on coaster". Unlike all previous works, PSALM shows how we can achieve 100% success rate on a certain type of task in the domain, by using LLMs to learn the action semantics from one example.

### LLM for PDDL Generation

There are two types of PDDL generation for LLM, generating the PDDL problem file or the domain file.

Generating the problem file requires to get access to the domain file, recent works such as LLM+P Liu et al. (2023), PDDLEGO Zhang et al. (2024) demonstrate the effectiveness of LLM to generate problem file for simple domains. The Planetarium Zuo et al. (2024) benchmark provides large-scale study of the LLM capability in generating PDDL problem files.

Unlike the success on generating problem file, LLMs, even the most powerful ones, performs badly on generating domain file Smirnov et al. (2024); Oswald et al. (2024). We formally define the domain induction problem, its evaluation measures, and provide a first-step solution PSALM to predicting the action semantics in the domain file in the text-based environments.

### LLM-based Memory Augmentation

Knowledge acquisition for task planning through dialog and web access has been studied in the past. Previous works Perera et al. (2015); Amiri et al. (2019) construct a knowledge base in an open domain to refer to for grounding user utterance. Recent works have shown LLM based memory augmentation to be effective. Sarch et al. (2023) builds a memory of language instruction and corresponding plan to retrieve from for prompting the LLM with a new instruction, where the retrieved interaction might inform planning. PSALM build probabilistic memory storing the belief for domain action semantics prediction.

## Appendix B Experiment Details

### Task Selection and Domain Continual Learning

For learning action semantics, we choose the first task that requires using all actions to reach the goal. Specifically, for Floortile, Barman, Termes and Tyreworlds domains, we choose task #1. For Grippers and BlocksW domains, we choose task #2. For Storage domain, we choose task #3, out of the 20 tasks.

If PSALM learns from a task that only requires partial actions to reach the goal. The planner may be able to generate successful plans for that tasks without knowing the entire domain. To overcome this issue, we also experiment on a setup where 

PSALM learns from task #1 for all domains and perform a continual learning setup. We report the learning process as below. Instead of directly learning the entire Storage domain from task #3 with 14 executed steps, we start the learning from task #1 with 13 executed steps, and recover 78.9% of the action semantics conditions. Then, based on what we learned from task #1, it takes 10 steps to recover 81.6% of the action semantics conditions learning from task #2, and 11 more steps to recover the entire domain from task #3. The continual learning process can be applied to real world exploration of certain domain as well. Thus, the "success without learning the entire domain" effect is not a limitation but a benefit of PSALM.

### Task Example Initial and Goal States

We provide the visualization of the initial and goal state for the selected training example per domain. The natural language description of the actions are from Liu et al. (2023).

BarmanFigure 7 shows the initial and goal state for task #1. The domain allows 12 actions: grasp a container, leave a container on the table, fill a shot glass with an ingredient, refill a shot class with an ingredient, empty a shot glass, clean a shot glass, pour an ingredient from a shot glass to a clean shaker, pour an ingredient from a shot glass to a used shaker, empty a shaker, clean a shaker, shake a cocktail in a shaker, and pour from a shaker to a shot glass.

BlocksW. Figure 8 shows the initial and goal state for task #2. The domain allows 4 actions: pick up a block from the table, put down a block on the table, stack a block on top of another block, unstack a block from on top of another block.

Floortile. Figure 9 shows the initial and goal state for task #1. The domain allows 7 actions: change the spray gun color paint the tile that is up from the robot, paint the tile that is down from the robot, move up, move down, move right, move left.

Grippers. Figure 10 shows the initial and goal state for task #2. The domain allows 3 actions: move from one room to another, pick up an object using the gripper, drop an object that it is carrying.

Storage. Figure 11 shows the initial state for task #3. The goal state is to move the crate to any location in the hoist store areas. The domain allows 5 actions: lift a crate using a hoist from a store area to an area, drop a crate from the hoist onto a surface in a store area, move a hoist from one store area to another connected store area, move a hoist from a store area to a transit area, move a hoist from a transit area to a store area.

Termes. Figure 11 shows the initial and goal state for task #1. The goal state shows the height of the blocks in each position. The domain allows 7 actions: move from a position to another, move up from a position to another, move down from a position to another, place a block at a neighboring position from the robot's current position, remove a block at a neighboring position from the robot's

\begin{table}
\begin{tabular}{l c r r r l r}  & **Goal** & \multicolumn{3}{c}{**Independent of**} & Role of LLM & Final planner \\ \cline{3-6}  & & Partial AS & Valid plans & Human eval & & \\ \hline Liu et al. (2023) & PF & ✗ & ✓ & ✓ & PF generator & - \\ Zhang et al. (2024) & PF & ✓ & ✓ & ✓ & PF generator & - \\ Hazra et al. (2024) & Plan & ✓ & ✓ & ✓ & Plan sampler & Symbolic + LLM \\ Valmeekam et al. (2023) & Plan & ✓ & ✓ & ✗ & Plan sampler & LLM \\ Lin et al. (2024) & Plan & ✓ & ✓ & ✗ & Plan sampler & Symbolic + LLM \\ Silver et al. (2023) & Program & ✗ & ✓ & ✓ & Code generator & LLM \\ \hline Arora et al. (2018) & AS & ✓ & ✗ & ✓ & - & - \\ Wong et al. (2023) & AS & ✗ & ✓ & ✓ & AS generator & Symbolic + LLM \\ Guan et al. (2023) & AS & ✓ & ✓ & ✗ & AS generator & Symbolic / LLM \\ Oswald et al. (2024) & AS & ✓ & ✓ & ✓ & AS generator & Symbolic \\ Han et al. (2024) & AS & ✓ & ✓ & ✗ & \begin{tabular}{r} PF generator + \\ AS generator \\ \end{tabular} & Symbolic \\ \hline PSALM & AS & ✓ & ✓ & ✓ & 
\begin{tabular}{r} Plan sampler + \\ AS generator \\ \end{tabular} & Symbolic \\ \hline \end{tabular}
\end{table}
Table 4: Comparing the PSALM domain induction task setup to representative related works in LLM and LLM-Modulo planning; here, “PF” is short for problem file and “AS” is short for action semantics. We consider the final planner using tree or heuristic search as symbolic.

 

current position, create a block at the depot, destroy a block at the depot.

TyreworldFigure 13 shows the initial and goal state description for task #1. The domain allows 13 actions: open a container, close a container, fetch an object inside a container, put an object into a container, loosen a nut on a hub, tighten a nut on a hub, jack-up a hub, jack-down a hub, unfasten a nut on a hub, fasten a nut on a hub, remove a wheel from a hub, put a wheel onto a hub, inflate a wheel using a pump.

### Other Hyperparameters and Details

The total cost of the API calls of PSALM over 7 domains is around SS80 for 1 run. We do not put the pure random baselines in Figure 4, because none of the pure random baselines complete any task within 1k iterations.

Besides, we only use the cheapest configuration of Fast-Downward, _i.e._, we are not using very smart symbolic solver. Fast-Downward has various advanced search configurations including using A* search (seq-opt-lmcut), using heuristics plus hill climbing algorithms (seq-sat-fd-autotune-1). We use the lama-first configuration, which is close to lazy greedy search designed to find solutions quickly without much regard for plan cost.

The IPC domains and tasks Seipp et al. (2022) are licensed under the MIT License. The Fast-Downward planner is distributed under the GNU GPLv2 license, while the VAL validator is licensed under the BSD 3-Clause, with copyright held by the University of Strathclyde, King's College London, and Schlumberger Ltd. Our use of the dataset and software complies with the terms of their respective licenses.

## Appendix C LLM Prompts

We list the prompt template for LLM trajectory sampler (Figure 14) and LLM action semantics predictor (Figure 15), and provide one complete example for each. Figure 16, 17 is the example for trajectory sampler, and Figure 18, 19 is the example for action semantics predictor 

Figure 8: BlocksW task #2 initial state (a) and goal state (b).

Figure 7: Barman task #1 initial state (a) and goal state (b).

Figure 9: Floortile task #1 initial state (a) and goal state (b).

 

Figure 12: Termes task #1 initial state (a) and goal state (b).

Figure 10: Grippers task #2 initial state (a) and goal state (b).

 

for those a job, a dump, a branch, a host, a node, i. node, i. first query, and a latest store.

The gate, some, numeric, and stored gives one to the next.

The last is calculated as in clusters.

The actual queue has not included.

The last works are for this case.

The main set of data or the most.

The new and features.

For each bit to collect Data were able to detect Event or the Web. Related Work should be divided into three subsets: the user should be at least one. The first two, more, better, and more created to fit the best, the best method is derived.

Figure 14: LLM trajectory sampler prompt template

Figure 13: Tyreworld task #1 initial state and goal state description.

 

* [[domain_nl]] The current guess of the preconditions and effects for actions are:
* [[action_description]] Please infer missing or correct wrong [[postfix]] for action [[action]] in CNF, given the PDDL types and predicates:
* [[domain_type_pred]] To help you predict the [[postifx]], we provide a running example in this domain. For the problem:
* [[task_nl]] The initial state of the task in PDDL is:
* [[task_init]] A plan (list of actions) runs in the simulator like:
* [[trajectory]] Please use the information from the simulator output carefully, and output the CNF in a clean format of "(and (...))", where each parentheses contains some predicates. Note that you should not specify types, e.g., "- object", in preconditions and effects.
* [[postifix]] for [[action]] are:

Figure 15: LLM action semantics predictor prompt template



* [156] M. C. Bader, J. D. Kiefer, and A. G. Riedl, "A new approach to the design of a multi-objective optimization system for a multi-objective optimization system in terms of applications," _Journal of Machine Learning Research_**2014**(3):1798-1803, 2014.

[MISSING_PAGE_POST]

 

Figure 17: LLM trajectory sampler prompt example (cont’)



* [16] M. C. Bader, R. J. Brouwer, and D. A. K. Sridharan, "Theoretical performance of the 2018-bit 

* [18] M. C. Bader, J. A. Harvey, and R. H. S. Heath, "The role of the \({}^{4}\)He-\({}^{3}\) reaction in a cell: The role of the \({}^{5}\)He-\({}^{6}\) reaction in a cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's cell's' cell's cell's cell's cell's'