VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning
Authors: Yichao Liang
Nishanth Kumar
Hao Tang
Adrian Weller
Joshua B. Tenenbaum
Tom Silver
Joao F. Henriques
Kevin Ellis
Venue: ICLR 2025 Spotlight (2025)
Keywords: learning abstractions for planning
neuro-symbolic ai
concept learning
Primary area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
TL;DR: learning neuro-symbolic predicates from interaction allows from improve sample efficiency, generalization and interpretability

Abstract: Broadly intelligent agents should form task-specific abstractions that selectively expose the essential elements of a task, while abstracting away the complexity of the raw sensorimotor space. In this work, we present Neuro-Symbolic Predicates, a first-order abstraction language that combines the strengths of symbolic and neural knowledge representations. We outline an online algorithm for inventing such predicates and learning abstract world models. We compare our approach to hierarchical reinforcement learning, vision-language model planning, and symbolic predicate invention approaches, on both in- and out-of-distribution tasks across five simulated robotic domains. Results show that our approach offers better sample complexity, stronger out-of-distribution generalization, and improved interpretability.

Published as a conference paper at ICLR 2025
VISUALPREDICATOR: LEARNING ABSTRACT WORLD MODELS WITH NEURO-SYMBOLI PREDICATES FOR ROBOT PLANNING
Yichao Liang', Nishanth Kumar", Hao Tang", Adrian Weller1.6, Joshua B. Tenenbaum',
Tom Silver', João João F. Henriques", Kevin Ellis?
'University of Cambridge, PCorell University, Massachusetts Institute of Technology, 'Princeton University, University of Oxford, The Alan Turing Institute
ABSTRACT
Broadly intelligent agents should form task-specific abstractions that selectively expose the essential elements of a task, while abstracting away the complexity of the raw sensorimotor space. In this work, we present Newro-Symbolic Predicates, first-order abstraction language that combines the strengths of symbolic and neu- ral knowledge representations. We outline an online algorithm for inventing such predicates and learning abstract world models. We compare our approach to hi- erarchical reinforcement learning, vision- Lansuanl model planning, and symbolic predicate invention approaches, on both and outoof-distrbbution tasks across five simulated robotic domains. Results show that our approach offers better sam- ple complexity, stronger out-of-distribution generalization, and improved inter- pretability.
INTRODUCTION
Planning and model-based decision-making for robotics demands an understanding of the world that both perceptually and logically rich. For example, a household robot needs to know that slippery objects, such greasy spatulas, are hard to grasp. Determining if the spatula is greasy is a subtle perceptual problem. As an example of logical richness, for robot to use a balance beam to weigh objects, it must count up the mass each side of the balance beam to determine which way the beam will tip. Counting and comparing masses are logically sophisticated operations.
In this work, show how efficiently learn symbolic abstractions that both perceptually and logically rich, and which can plug into standard robot task-planners to solve long-horizon tasks. We consider a robot that encounters new environment involving novel physical mechanisms and new kinds of objects, and which must learn how to plan in this new environment from relatively few environment interactions (the equivalent of minutes or hours of training experience). The core of our approach is to learn an abstract model of the environment in terms of Neuro-Symbolic Predi- cates (NSPs, see Fig. [T.. which are snippets of Python code that can invoke -sngun-l models (VLMss) for querying perceptual properties, and further algorithmically manipulate those properties using Python, in the spirit of ViperGPT and VisProg (Surís et al. 2023 Gupta & Kembhavi] (022).
In contrast, traditional robot task planning uses hard-coded symbolic world models that cannot adapt to novel environments (Garrett et al. 2021: Konidaris 2019). Recent works pushed this direction with limited forms of learning that restrict the allowed perceptual and logical abstractions, and which further require demonstration data instead of having the robot explore on its own (Silver et al. 2023 Konidaris et al.) 2018). The representational power of Neuro-Symbolte Predicates allows much broader set of perceptual primitives (essentially anything a VLM can perceive) and also deeper logical structure (in principle, anything computable in Python).

---

Published as a conference paper at ICLR 2025
(A) making and pouring coffee into of cups
example learned predicate-planning operator
1: sperator deforation, Turmón option.
option: Plocennsschinejgg machine)
option: TurnOn(machine)
option: Pounjjug tup1)
(def-op rel.coofeeescccheee nochire 7x1:3ng (rttre
iprecondition (lagteMochine 2xt 7x8)
:pentcordittinn (Cugfilled hx2:5u83)
neuro symoolic medicatts
def SugTnHechse<(etatt, atjacte]:
1F the 31E hels, 1t conter be the sachtne. if teising(state, (areiettttt,eettteeeee (ww.:ettttt.... objecta (rebot_ ()))))))))))))))
abstract state
abstract state
abstract state
Holdinglett-ag!
jugünMachinejjg. machinet GrtpperOpent
Jugfille(Jug Holdinglug(jzz)
Crop to Fecus en 1kg and coffee machine.
wetteeccee_pe_eeeeeee [___jects(((344... sackine) neturn caste.qurry. VUM
+" mume is placed inside (machine. mtm.eiee.d.mm)
(B) using balance beam to activate machine that requires balanced platters
example learned wvedicate-planning operator
sperator dertastion, Pressductio option..
option: Rickblock3)
option: FutDnFlatesplatee))
PressButton
(def (hxtonechiie hxtiplate tiziplate ixtert
heioplate hxtoplate)
sprecondation (Sistributeitvelly hxz hx)
iportconittion (Pschtneön Tel:nochtne))
Nearo Symboldc Predicute.
def esst-itureeeeellyeee otjects):
if plate1 platez: return Palse counti, countà
abstract state
abstract state
abstract state
ston atoms:
DirectlyOmibleda, block2) Onelotetblock3, plate2)
Hokdnggblocka
Detributedivelllplatt platez) GripperOpent)
if intn.prretccs OnFlate:
Clear(biock2)
GripeerOpen)
DivecttyOnPlanettiico.. plate?)
GnPlatichiockk, plate1)
elff stan.orjects(1] platez: count2
count2
count2
Figure 1: Robot learning domains illustrating learned Neuro- Symbolic predicates. In (A) we learn a predicate that queries a VLM to check if a coffee jug inside coffee machine. In (B) we learn a predicate that checks if balance beam is balanced. (Code lightly refactored better fit in figure.)
Yet there are steep challenges when learning Neuro-Symbolic Predicates to enable effective plan- ning. First, the predicates must be learned from input pixel data, which is extremely complex and potentially noisy. Second, they should not overfit to the situations encountered during training, and instead zero-shot generalize to complex new tasks at test time. Third, we need an efficient way of exploring diffferent possible plans collect the data needed to learn good predicates. To address these challenges we architect a new robot learning approach that interleaves proposing new predi- cates (using VLMs), predicate scoring/validation (adapting the modern predicate-leariing algorithm by SSilver et al. ).02)), and goal-driven exploration with a planner in the loop. The resulting archi- tecture is then able successfully learn across five different simulated environments, and is more flexible and more sample-efficient compared to competing neural, symbolic, and LLM baselines.
We highlight the following contributions: (1) NSPs, a state representation for decision- making using both logically and perceptually rich features; (2) An algorithm for inventing NSPs by interacting with an environment, including an extension new operator learning algorithm; and (3) Evaluation against 6 methods across 5 simulated robotics tasks.
2 PROBLEM FORMULATION
We consider the problem of learning state abstractions for robot planning over continuous state/ac- tion spaces, and doing so from online interaction with the environment, rather than learning from human-provided demonstrations. We assume a predefined inventory of basic motor skills, such as pick/place, and also assume a basic object-centric state representation (explained further below), which is common assumption (Kumar et al.) 2024 Silver et al.) 2023 2022). The goal is to learn state abstractions from training tasks that generalize to held-out test tasks, enabling the agent to solve many test tasks possible while using minimal planning budget.
Tasks. task T is a tuple (O, ro- g) of objects O, initial state Ty. and goal The allowed states depend the objects O, so write the state space as Xo (or just X' when the objects are clear from context). Each state x E Xo includes raw RGB image and associated object features, such 3D object position.
2

---

Published as a conference paper at ICLR 2025
Environments. Tasks oceur within enviromment E, which is tuple f. where U C R is a low-level action space (e.g. motor torques), C is a set of controllers for low-level skills (e.g. pick/place), f: X x u X' is a transition function, and A is a set of object types (possible outputs of an object classifier). The environment is shared across tasks.
Built-in Motor Skills. We assume skills C, each of which has parameters that abstract over which object(s) the skill acts on. For example, the agent can apply a skill such as Place (bblock1, ?block2) to stack any pair of blocks atop one another, where block is a type in A. We assume the agent can determine whether a skill has been successfully executed upon completion. Skills can be modeled within the options framework (Sutton al.) 1999). The skills and the objects O induce an action space Ao (or simply A when the context is clear).
Skills, tasks, and environments are the primary inputs to our system. The primary outputs- what we actually learn are higher-level abstractions over these basic states and actions.
Abstracting the State. A predicate sb is a Boolean feature of state, which can be parametrized by specific objects in that state. We treat this as function /! Om - (X' + B) that an indicator, given m objects, of whether a predicate holds On (?block1, ?bl ock2) inputs pair of blocks, and outputs state classifier for whether the first block atop the second block. set of predicates vi induces an abstract state corresponding to all the predicate/object combinations that hold in the current state:
Predicates:
state. For example, the predicate
ABSTRACT (z) = ((V, ((v,0).... oI.... 0m) (b,,,.... )m) holds in state x, for vp E V and oj E OO} (I)
We write S for the set of possible abstract states.
High-Levvel Actions: Refining the action space! T Planning requires predicting how each skill
transforms the abstract state representation. To make these predictions, High-Level Actions (HLAs) augment skills with preconditions specifying which abstract states allow successful use of that skill, and postconditions, specifying how the skill transforms the abstract state. Like predicates, an HLA is parametrized by the specific objects it acts upon. Formally, an HLA w is function from tuple of objects in O to a tuple (,,, PRE, EFF EFF where π E Ao is a skill, PRE is the precondition, and the postcondition consists of EFF (predicates added to the abstract state) and EFF (predicates removed from the abstract state).
As an example of an HLA, consider PlaceOnTable (?block, ?table, ?underBlock), with PRE = {Clear ( (?block) J. ErF+ = {On (?block, ?table)), and EFF {on (?block, ?underBlock) }. using skill place (?block, ?table). This means placing block on table, which was previously on top of underBlock, causes the block to be on the table, and no longer on top of underBlock. This HLA is formally a function with arguments Pblock, ?table, ?underBlock.
HLA Notation. We write for the set of HLAS (what the agent learns), and Go for their instan- tiations on objects O (how the agent uses them in particular task). We use the variable w for HLAs, so we would write w E Ω. We use w for HLAs applied to particular objects, so we'd write # (I, PRE, EFF EFF E ENO
A Abstract State Transitions. The predicates and HLAs together define an abstract world model, whose transition function F : S x lo S is
S EFFt EFF if PRE C otherwise
F (s.= LI, PRE. EFF EFF
undefiiedd
Having learned predicates and high-level actions, we then solve problems by hierarchical planning:
low-level plan is a sequence of n skills applied to objects (m.,..., Fn) E AB. It solves a task with goal y and initial state zg if sequencing those skills starting from 20 satisfies g.
hivh-level plan a sequence of n. HI.As applied to objects, ,.........
A note on types. Because the environment provides object types, we augment predicates and HLAS with typing information for each object-valued argument. Equivalently, predicates return false, and skills terminate immediately with failure, when applied to arguments of the wrong type.
In the planning literature, High-Levvel Actions are also sometimes called operators. In the planning literature, called lifted operator, while u would be grounded operator.
3

---

Published as a conference paper at ICLR 2025
3 NEURO-SYMBOLLE PREDICATES
Neuro-Symbolic Predicates (NSPs) represent visually grounded yet logically rich abstractions that enable efficient planning and problem solving. As Figure illustrates, these predicates are neuro- symbolic because they combine programming language constructs (conditionals, numerics, loops and recursion) with API calls to neural vision-language models for evaluating
natural language assertions. NSPs can be grounded in visual perception, and also proprioceptive and object-tracking features, such as object poses, common in robotics (Kumar et al. 2024 2023b) Curtis et al. 2022; 2024b). We consider two classes of NSPs: primitive and derived. Primitive NSPs evaluated directly on the raw state, such as Holding ng (obj) (which would use VLM queries) or Gr Grippers opperpeen () (which would use proprioception). Derived NSPs instead determine their truth value based on the truth value of other NSPs, analogous derived predicates in planning (Thiébaux et al. 2005 McDermott et al. 1998).
Primitive NSPs. We provide Python API for computing the raw state, including the ability to crop the image to particular objects and query VLM in natural language. See Appendix
Derived NSPs. Instead of querying the raw state, derived NSP computes its truth value based only on the truth value of other NSPs. Derived NSPs handle logically rich relations, such as OnP late in fig. 2 which recursively computes if block is on plate, or on something that is on plate.
Evaluating Primitive NSPs. No VLM is 100% accurate, even for simple queries like "is the robot holding the jug?", especially in partially observable environments. To increase the accuracy and precision of NSPs, we take the following two measures (see Appendix BB. for an example prompt).
First, because single image may not uniquely identify the state (e.g. due to occlusion), we provide extra context to VLM queries. Consider a robot whose gripper is next to jug, but whose own arm occludes the jug handle, making it uncertain whether the jug is held by the gripper or merely next to it. Knowing the previous action (e.g. Pick ( (jug)) ) helps resolve this uncertainty. We therefore further condition NSPs on the previous action, as well as the previous visual observation (immediately before the previous action was executed) and previous truth values for the queried ground atom.
Second, we visually label each object in the scene by overlaying unique ID number each object in the RGB image (following Yang et al. 2023). That way, to evaluate for example Ho Holding (block2), can query VLM with "the robot is holding block2", where block2
def Holding(state: RawState, objects: Sequmnc[(bjjcct] <> baol:
" the robot holding the block,"
block, objects # The black can't be held if the robot's hand is open. robot state.get, geteeeeeet.ojjtstttteeee.. porects_rrbot_typee if state.get(rabos, "fingers") >e 0.5:
return False
attention, Inage crrate.cop.tti. objectst robot]] return cooluate_sinptee assertion( [blockk id, nare) is held by the robot", attention inage)
def OnPlate(atons: Set[GroundAton), objects: (rruence((bjett]] bool:
"hehhee block directly transitively plate ***
block, plate objects
for aton in atons:
aton. aton.predicate DirectlydnPlate and atan.ebjects [block, plate]: return True
sther_blocks (a.objects(8)
a.predicate Directly0n or a.predicate D(rectlyOnPlate)
for other_black in other_blocks:
holdsl False for ston in ators:
if predicced DirectlyOn and aton. abjecobjects [black, ather_block];
holds] True
break
holds1 and OnPlate(atons, [other_ bother_block, plate]): return True
return
Figure 2: Example classifiers for Holding and OnP late NSP.
4

---

Published as a conference paper at ICLR 2025
is labeled with "2." This disambiguates the objects in scene, allowing an NSP to reason precisely about which block is held, rather than merely represent that some block is held.
How Derived NSPs interact with HLAss HLAs form abstract world model that predicts which predicates are true after performing a skill (the postcondition). Derived predicates do not need to occur in the postcondition, because based on the predicted truth values of primitive NSPs. Therefore, HLAs can have derived predicates in the precondition, but never in the postcondition.
immediately calculate which derived predicates true
4 HIERARCHICAL PLANNING
We use the learned abstract world model to first make a high-level plan (sequence of HLAs), which then yields a low-level action sequence by calling the corresponding skill policy for each HLA. High-level planning leverages widely-used fast symbolic planners, which, for example, conduct A search with automatically-derived heuristics (e.g. LM-Cut, Heimert Domshlak 2009).
However, there may be mismatch between high-level plan, which depends on potentially flawed abstractions, and its actual implementation the real world. Learning is driven by these failures. More precisely, hierarchical planning break down in one of two ways:
Planning Failure #1: Infeasible. A high-level plan is infeasible if one of its constituent skills fails to execute.
Planning Failure #2: Not satisficing. high-level plan is not satisficing its constituent skills successfully execute, but do not achieve the goal.
When solving a task we generate a stream of high-level plans and execute each one until a satisficing plan (achieving the goal) is generated, until hitting planning budget Nabstract.
LEARNING AN ABSTRACT WORLD MODEL FROM INTERACTING WITH THE ENVIRONMENT
Algorithm Online Pred. Invention(E, T, Vo, So D)
Algorithm [ shows how interleave
1: init: Prest -00, best solve rate
learning predicates (state abstraction),
2: init: Phest 30, best number of failed plans
learning HLAs (abstract transition func- tion), and interacting with the environ- ment. The learner takes in an en- vironment ε, set of training tasks T, an initial predicate set Vo (which
3: init: V + Vio
4: for i E range(1, Rmmn ise do
5:
Di, Pr; Mi +- Explore( i- 94-1, T) b. section 3.1
6:
if Poest (ps = Paea and < Voes) then V. S2 Pr, vi
7:
Vhsre Uresa, Pren, Uhest
usually the goal predicates), an ini- tial set of HLAs Oo (which are largely
8:
if ====0 0 then
break
10:
D DUDD
empty, section and an initial dataset D (empty, except when doing transfer learning from earlier environments). It tracks its learning progress using Phest. the highest training solve rate, and Whest the lowest number of infeasible plans.
11:
if pi S pi (p pi-1 and 24 > v1-1) then
12:
V + Propose NSPs(D, 1. b section 5.2
13:
V; +- Select Predicates(D, V/( b section
5.3
14:
Ω. <- Learn HighLevelActions(( v.
>> section 3.4
15:
return Weerr, PreSt
5.1 EXPLORATION
Our agent explores the environment by planning with its current predicates/HLAs and executing the plans. The agent is initialized with underspecified, mostly empty HLA(s) (that is, the preconditions and effects mostly empty sets, except with goal predicates in the effects if appropriate, so that the planner can generate plans) It collects data by trying to solve the training tasks (generate and execute abstract plans until the task is solved Nabstract plans are used, as described in section 4 and collects positive transition segments (from successfully-execute skills), negative state-action tuples (from skills that failed to execute successfully) and satisficing plans, if any.
A Iternatively, it could perform exploration through random skill selection.
5

---

Published as a conference paper at ICLR 2025
5.2 PROPOSING PREDICATES
We introduce three strategies for prompting VLMs to invent diverse, task-relevant predicates two that are conditioned on collected data, and one that is not (see Appendix further details).
Strategy #1 (Discrimination) helps discover predicates that are good preconditions for the skills. We prompt a VLM with example states where a skill succeeded and failed, and ask it to generate code that predicts when the skill is applicable.
Strategy #2 (Transition Modeling) helps discover predicates helpful for postconditions. We prompt VLM with before (or after) snapshots of successful skill execution, and ask it to gener- ate code that describes properties that changed before (or after, respectively).
Strategy #3 (Unconditional Generation) prompts VLMs to propose new predicates logical ex- tensions of existing ones (whether built-in or previously proposed), without conditioning on the raw planning data. This approach helps create derived predicates.
5.3 SELECTING A PREDICATE SET
VLM-generated predicates typically have low precision-not all generations are useful or sensible- and too many predicates will overfit the model to what little data it has collected. One solution could be the propose- propose-then-seles then-select paradigm (Silver et al. 2023). [Silver et al.] (2023) propose an effective predicate selection objective but requires around 50 expert plan demonstrations. We assume no demonstration data, and in general, we might not find any satisficing plans early in learning- Therefore we need new way of learning from unsuccessful plans.
To address this, we devise novel objective that scores a set of predicates T based on classification accuracy, plus a simplicity bias. The objective score is obtained by first learning HLAs using the set of predicates the HLAs (see Appendix B.3).. Later in learning. after discovering enough (a hyperparameter one can choose) satisficing plans, we switch to the objective of Silver et al. (2023), which takes planning efficiency and simplicity into account.
(discussed more in section [.4). and then computing the classification accuracy of
We perform a greedy best-first search (GBFS) with either score function as the heuristic. It starts from the of goal predicates V and adds single new predicate from the proposed candidates at each step, and finally returns the set of predicates with the highest score. This effectively selects V; f arg minggg(w) (VV),, where J-)) is the objective function. In experiments, we found the search space is small enough that the enumeration typically takes just a few minutes on a single CPU. For larger search spaces, local hill climbing could be used in place of GBFS.
5.4 LEARNING HIGH-LEVEL ACTIONS
We further learn high-level actions Ω, which define an abstract transition model, in the learned predicate space, from interactions with the environment. We follow the cluster and intersect operator learning algorithm (Chitnis et al.] 2022) and improve its precondition learner for more efficient exploration and better generalization. Chitnis et al. (2022) assume given demonstration trajectories and learns restricted preconditions so that the plans are most similar to the demonstrations. Our agent explores the environment from scratch and does not have demonstration data follow restrictivelyy On the other hand, our agent needs more optimistic world models to explore unseen situations to solve the task. Our precondition learner ensures that each data the transition dataset modeled by one and only one high-level action and minimizes the syntactic complexity of the HLA to encourage optimistic world models. See Appendix B.2 details.
6 EXPERIMENTS
We design our experiments to answer the following questions: (Q1) How well does our NSP rep- resentation and predicate invention approach compare to other otate-of-the-ar methods, including popular HRL or VLM planning approaches? (Q2) How do the abstractions learned by method perform relative to manually designed abstractions and the abstractions before any learning? (Q3) How effective is NSP representation compared to traditional symbolic predicates, where classi-

---

5. Balance. In this environment, the agent tasked with turning on a machine by pressing a button in front of it, but without prior knowledge the mechanism required activate (in this case, balancing an equal number of blocks on both sides). The agent has access to a PressButton skill, blocks, while test Published as a conference paper at ICLR 2025
fiers based manually selected object features? (Q4) What is the contribution of our extended operator learning algorithm to overall performance?
Experimental Setup. We evaluated seven different approaches across five robotic environments simulated using the PyBullet physics engine (Coumans & Bai 2016). Each result averaged over five random seeds, and for each seed, sample 50 test tasks that feature more objects and more complex goals than those encountered during training. The agent is provided with training tasks in the Cover and Coffee environments, 10 tasks in Cover Heavy and Balance, and 20 tasks in Blocks. The planning budget Vlabstract is set to 8 for all domains except Coffee, where it is set to 100. The approaches are run on single CPU except for the MAPLE baseline, which utilizes uses GPU.
Environments. We briefly describe the environments used, including their hand-coded closed-looy controllers, which are shared across all approaches. Additional details can be found in appendix D
1. Cover. The robot is tasked with picking and placing specific blocks to cover designated regions the table, using Pick and Place skills. Training tasks involve blocks and 2 targets, while test tasks increase the difficulty with 3 blocks and 3 targets.
2. Blocks. The robot must construct towers of blocks according to specified configuration, using Pick, Stack, and PlaceOnTable skills. The agent trained on tasks involving blocks and tested on more challenging tasks with 5 or 6 blocks.
3. Coffee. The robot is tasked with filling cups with coffee. This involves picking up and placing a jug into a coffee machine, making coffee, and pouring it into the cups. The jug may start at random rotation, requiring the robot rotate it before it can be picked up. The environment provides 5 skills: Twist, Pick, Place, TurMMaccineOn, and Pour. Training tasks involve filling cup, while tasks require filling 2 or 3 cups.
4. Cover Heavy. This is variant Cover with "impossible tasks" which asks the robot to pick and placing white marble blocks that are too heavy for it to pick up. The environment retains the same controllers
number of objects as the standard Cover environment. An impossible task is considered correctly solved if the agent determines that the goal is unreachable with its existing skills (i.c., no feasible plan can be generated).
along with the skills from the Blocks domain. Training tasks involve tasks increase the difficulty with 4 or 6 blocks.
Approaches. We compare our approach against 5 baselines and manually designed state abstraction.
1. Ours. Our main approach.
Cover
Blocks
Coffee
Cover Heavy
Balance
Train Tasks
Eval. Tasks
Figure 3: Environments. Top train task examples. Bottom evaluation task examples.

---

Published as a conference paper at ICLR 2025
Cover
Blocks
Coffee
Cover Heavy
Balance
Oradie
MAPLE
Via
5m. prod.
op
No mvort
25
90
75
100
25
50
75
100
25
50
75
100
25
53
75
100
25
100
Oracle
0us
Abrate
50 75
120
25
50
75
100
25
50
75
100
50
75
100
100
Figure 4: Top row: percentage solved for different domains (1). Bottom: percentage of planning budget used to find the satisficing plans (J). The dashed line shows the minimal number of plans needed to solve all the tasks (1 plan per task).
2. MAPLE. a HRL baseline that learns to select high-level action by learning Q function, but does not explicit learn predicates and perform planning. This is inspired by the recent work MAPLE (Nasiriany et al. (002b)). While we have extended the original work with the capacity of goal-conditioning. the implementation is still not able deal with goals involving more objects than it has seen during training. Hence, we are only evaluating this approach with tasks from the training distribution.
3. ViLa (Hu et al. 2023). VLM planning baseline which zero-shot prompts a VLM to plan sequence of actions, without learning.
4. Sym. pred. A baseline that uses the same online learning algorithm but only has access to object features that are commonly present in roboties tasks when writing predicates, i.e., without open- ended VI.M queries and derived predicates. This shares a similar representation as recent work Interpret (Han et al. 2024 but is still distinct since they mostly learn from human instruction.
5. Ablate op. An ablation that does not use our extension to the operator learner.
6. No invent. A baseline that uses the abstractions our approach is initialized with and does not perform any learning.
7. Oracle. An "oracle" planning agent with manually designed predicates and operators.
Results and Discussion. Figure presents the evaluation task solve rate and the planning budget utilized. Examples of an online learning trajectory with invented predicates, instances learned abstractions, and further planning statisties (such node expanded and walltime) provided in appendix
Our approach consistently outperforms the HRL and VLM planning baselines, MAPLE and ViL.,, across all tested domains, achieving near-perfect solve rates (Q1). With similar amounts of inter- action data, MAPLE struggles to perform well, even on tasks within the training distribution. This limitation could potentially be mitigated with significantly larger datasets, though this is often im- practical in roboties due the high cost of real-world interaction data and the sim-to-real gap in transferring simulation-traine consistent with recent observations (Kambhampati
policies. ViLa demonstrates limited planning capabilities, which is
al., 2024). While performs adequately on simple tasks like Cover, where the robot picks and places blocks, its performance drops significantly when blocks are initialized in the robot's grasp, as it tends to redundantly attempt picking actions. This behavior suggests overfitting, In more complex domains, ViLa often generates infeasible plans, such as attempting to pick blocks from a stack's 's middle or trying to grasp a jug without consider- ing its orientation. We think introducing demonstrations incorporating environment interactions could potentially alleviate these issues.
Our approach significantly outperforms No invent. demonstrating the clear benefits of learning pred- icate abstractions over relying on initial underspecified representations. achieves similar solve rates and efficiency to the Oracle baseline, which uses manually designed abstractions (Q2). This
8

---

Published as a conference paper at ICLR 2025
underscores the ability of our method to autonomously discover abstractions effective as those crafted by human experts.
Addressing (Q3), while Sym. pred. performs well in simple domains like Cover, it struggles to invent predicates that require grounding in perceptual cues not explicitly encoded in object features. For instance, in Coffee, it cannot reliably determine if a jug is inside coffee machine based on object poses-a key precondition for the TurnMachin neon action. Similarly, in Cover Heavy, it fails to recognize blocks that are too heavy to lift, which is critical for identifying unreachable goals. Additionally, without derived NSPs, reasoning accurately and efficiently about abstract concepts in the abstract world model (such as whether the number of blocks on both sides of balance equal) becomes challenging, which is critical for solving Balance More generally, we hypothesize that providing all feature-valus pairs for every object in each state during prompting overwhelms existing VLMs, leading to poor predicate invention. This likely accounts for the subpar performance, even in simple domains like Blocks. These limitations emphasize the strengths of our NSP representation and learning pipeline.
Finally, to answer (Q4), we find that our approach performs better than Ablate op., which sometimes learns unnecessarily complex preconditions that overfit early, limited data, hindering further learning on training tasks. In other cases, overly specific preconditions result in good training performance but poor generalization, such requiring JugInMachine for the Pour action. This demonstrates the value of our operator learner, especially in data-scarce, exploration-base learning settings.
RELATED WORKS
Hierarchical Reinforcement Learning (HRL) HRL tackles the challenge of solving MDPs with high-dimensional state and action spaces, common in robotics, by leveraging temporally extended, high-level actions (Barto & Mahadevan 2003). The Parameterized Action MDPs (PAMDPs) frame- work (Masson et al. (016)) builds this by interaaing discrete actions with continuous parameters, optimizing both the action and its parameterization using the Q-PAMDP D-PAMiDaaloorithn. MAPLE (Nasiri- any et al. 2022a) further builds on this by using library of behavior primitives, such as grasping and pushing, combined with a high-level policy that selects and parameterize these actions. We implement a version of this with the extension of goal-conditioned high-level policy as baseline. Generative Skill Chaining (GSC) (Mishra et al., (023) further improves long-horizon planning by using skill-centric diffusion models that chain together skills while enforcing geometric constrains. Despite these advancements, they still face challenges in sample complexity, generalization, and interpretability.
Large Pre-Trained Models for Robotics With the rise of large (vision) language models (VLMs), many works explore their application in robotic decision making. RT-2 (Brohan et al., 2023) treats robotic actions as utterances in an "action language" learned from web-scale datasets. SayCan and Inner Monologue (Ahn et al. 2022 Huang et al. 2022) use LLMs to select skills from a pretrained library based on task prompts and prior actions. Code Policy (Liang al., 2023) prompts LLMs to write policy code that handles perception and control. Recent works extend this to bilevel planning
(Curtis et al., 2024a), but do not learn new predicates. ViL.a (Hu et al. 2023) queries VL.Ms for action plans, executing the first step before replanning. We implement an open-loop version of Vil.a to compare with its initial planning capabilities.
Learning Abstraction for Planning Our work builds on rich body of research focused on learning abstractions for planning. Many prior works have explored offline methods such as learning action operators and transition models from demonstrations using existing predicates (Silver et al., 202T Chitnis et al., 2022] Pasula et al. 2007 Silver al. [0022] Kumar al., 2023a). While Silver et al. (2023) explore learning predicates grounded in object-centric features, our approach goes further by inventing open-ended, visually and logically rich concepts, without relying on hand-selected fea- tures. Additionally, unlike their demonstration based approach, ours learns purely online. Konidaris ctt al. (2018) and consequent works (James et al. 2022; 2020) discover abstraction in an online fashion by leveraging the initiable and terminations set of operators that satisfy an abstract subgoal property. James et al.] (2020) incorporate an egocentric observation space to learn more portable representations, and James et al. (2022) define equivalence of options effects on objects to derive object types for better transferability. Nevertheless, they work on a constrained class classifiers (such decision trees or linear regression with feature selection), which limits the effectiveness and

---

Published as a conference paper at ICLR 2025
generalizability of learned predicates. Kumar al. (2024) performs efficient online learning, but focuses on sampler learning rather than predicate invention.
8 CONCLUSION
In this work, we introduced Neuro-Symbolic Predicates (NSPs), novel representation that com- bines the flexibility of neural networks to represent open-ended, visually grounded concepts, and the interpretabilityy and compositionality of symbolic representations, for planning. To support this, we developed an online algorithm for inventing NSPs and learning abstract world models, which allows efficient acquisition of NSPy. Our experiments across five simulated robotic domains demon- strated that our method outperforms existing approaches, including hierarchical reinforcement learn- ing, VLM planning, and traditional symbolic predicates, particularly in terms of sample efficiency, generalization, and interpretability. Exciting areas for future work include incorporating recovery mechanisms for failed plans, enhancing exploration efficiency, scaling to partially observable and real-world domains, and relaxing assumptions about skills leverage advances in policy synthesis Ma et al. 2023). or motion planning (Huang et al.,
[Liang al. 2023). RL (Liang al. 2024 2024).
REFERENCES
Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. Do as can, not as say: Grounding language in robotic affordances. arXiv preprint arXivv2204.0169/ 2022.
Andrew G Barto and Sridhar Mahadevan. Recent advances in hierarchical reinforcement learning. Discrete event dynamic systems, 13:341-378 2003.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choroman- ski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. Rt-2: Vision-language-nn models transfer web knowledge to robotic control, arXiv preprint arXiv:2307. (50.:/3887..881888 2023.
Rohan Chitnis, Tom Silver, Joshua B Tenenbaum, Tomas Lozano-Perez, and Leslie Pack Kaelbling. Learning neuro-symbolic relational transition models for bilevel planning. 2022 IEEE/RSJ International Conference Inielligent Robots and Systems (IROS), pp. 4166-4173. IEEE, 2022.
Erwin Coumans and Yunfei Bai. Pybullet, python module for physics simulation for games, robotics and machine learning, 2016.
Aidan Curtis, Xiaolin Fang, Leslie Pack Kaelbling, Tomás Lozano-Pérez, and Caelan Reed Garrett. Long-horizon manipulation of unknown objects via task and motion planning with estimated affordances. In 2022 International Conference Robotics and Automation (ICRA). pp. 1940- 1946. IEFF, 2022.
Aidan Curtis, Nishanth Kumar, Jing Cao, Tomás Lozano- Pérez, and Leslie Pack Kaelbling. Trust the proc3s: Solving long- horizon robotics problems with Ilms and constraint satisfaction, 2024a.
Aidan Curtis, George Matheos, Nishad Gothoskar, Vikash Mansinghka, Joshua Tenenbaum, Tomás Lozano-Pérez, and Leslie Pack Kaelbling. Partially observable task and motion planning with uncertainty and risk awareness. arXiv preprint arXiv:2403.10455 2024b.
Caelan Reed Garrett, Rohan Chitnis, Rachel Holladay, Beomjoon Kim, Tom Silver, Leslie Pack Kaelbling, and Tomás Lozano-Pérez. Integrated task and motion planning. Annual review of control, robotics, and autonomous systems, 4(1):265-293, 2021.
Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning without training. ArXiv, abs/221 15559 2022.
Muzhi Han, Yifeng Zhu, Song-Chun Zhu, Ying Nian Wu, and Yuke Zhu. Interpret: Interac- tive predicate learning from language feedback for generalizable task planning. arXiv preprint arXiv:2405. 19758. 2024.
10

---

Published as a conference paper at ICLR 2025
Malte Helmert and Carmel Domshlak. Landmarks, critical paths and abstractions: what's the dif- ference anyway? In Proceedings of the International Conference on Automated Planning and Scheduling, volume 19, pp. 162-169, 2009.
Yingdong Hu, Fanqi Lin, Tong Zhang, Li Yi, and Yang Gao. Look before you leap: Unveiling the power of gpt-4v robotic vision- language planning. arXiv preprint arXiv:2311.17842, 2023.
Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, et al. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608 2022.
Wenlong Huang, Chen Wang, Yunzhu Li, Ruohan Zhang, and Li Fei-Fei. Rekep: Spatio- temporal reasoning of relational keypoint constraints for robotic manipulation. arXiv preprint arXiv:2409.01652, 2024.
Steven James, Benjamin Rosman, and George Konidaris. Learning portable representations for high-level planning. In International Conference Machine Learning, pp. 4682-4691. PMLR, 2020.
Steven James, Benjamin Rosman, and GD Konidaris. Autonomous learning of object-centric ab- stractions for high-level planning. In Proceedings of the The Tenth International Conference on Learning Representations, 2022.
Subbarao Kambhampati, Karthik Valmeekam, Lin Guan, Mudit Verma, Kaya Stechly, Siddhant Bhambri, Lucas Saldyt, and Anil Murthy. Llms can't plan, but can help planning in Ilm-modulo frameworks, 2024.
George Konidaris. On the necessity of abstraction. Current opinion in behavioral sciences, 29:1-7, 2019.
George Konidaris, Leslie Pack Kaelbling, and Tomas Lozano-Perez. From skills to symbols: Learn- ing symbolic representations for abstract high-level planning. Journal of Artificial Intelligence Research, 61:215-289, 2018.
Nishanth Kumar, Willie McClinton, Rohan Chitnis, Tom Silver, Tomás Lozano-Pérez, and Leslie Pack Kaelbling. Learning efficient abstract planning models that choose what predict. In Conference on Robot Learning, pp. 2070-2095. PMLR, 2023a.
Nishanth Kumar, Willie McClinton, Kathryn Le, and Tom Silver. Bilevel planning for robots: An illustrated introduction. 2023b. https:///lis.csnt wec.uhiieee--paannng---ooooooooo introduction.
Nishanth Kumar, Tom Silver, Willie McClinton, Linfeng Zhao, Stephen Proulx, Tomás Lozano- Pérez, Leslie Pack Kaelbling, and Jennifer Barry. Practice makes perfect: Planning to learn skill parameter policies, 2024.
Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. 2023 IEEE International Conference Robotics and Automation (ICRA), pp. 9493-9500. IEEE, 2023.
Yichao Liang Kevin Ellis, and João Henriques. Rapid motor adaptation for robotic manipulator arms. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 16404-16413, 2024.
Yecheng Jason Ma, William Liang. Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayara- man, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Eureka: Human-level reward design via coding large language models. arXiv preprint arXiv:2310. 12931, 2023.
Warwick Masson, Pravesh Ranchod, and George Konidaris. Reinforcement learning with param- eterized actions. Proceedings of the AAAI conference on artificial intelligence, volume 30, 2016.
11

---

Published as a conference paper at ICLR 2025
Drew McDermott, Malik Ghallab, Adele E. Howe, Craig A. Knoblock, Ashwin Ram, Manuela M. Veloso, Daniel S. Weld, and David E. Wilkins. Pddl-the planning domain definition language. 1998. UtLtstss::/ www.ttpp....aaaaaaaa.aa....aa..a www.etttitnntoooeeo.ee wtt.:/aa.......ooaa....:.5..000.
Utkarsh Aashu Mishra, Shangjie Xue, Yongxin Chen, and Danfei Xu. Generative skill chaining: Long-horizon skill planning with diffusion models. In Conference on Robot Learning, pp. 2905- 2925. PMLR. 2023.
Soroush Nasiriany, Huihan Liu, and Yuke Zhu. Augmenting reinforcement learning with behavior primitives for diverse manipulation tasks. In 2022 International Conference Robotics and Automation (ICRA), pp. 7477-7484. IEEE, 2022a.
Soroush Nasiriany, Huihan Liu, and Yuke Zhu. Augmenting reinforcement learning with behavior primitives for diverse manipulation tasks. In 2022 International Conference Robotics and Automation (ICRA), pp. 7477-7484. IEEE, 2022b.
Hanna M Pasula, Luke S Zettlemoyer, and Leslie Pack Kaelbling. Learning symbolic models of stochastic domains. Journal of Artificial Intelligence Research, 29:309-355 -52,, 2007.
Tom Silver, Rohan Chitnis, Joshua Tenenbaum, Leslie Pack Kaelbling, and Tomás Lozano-Pérez. Learning symbolic operators for task and motion planning. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3182-3189, IEEE, 2021.
Tom Silver, Ashay Athalye, Joshua B Tenenbaum, Tomas Lozano-Perez, and Leslie Pack Kaelbling. Learning neuro-symbolic skills for bilevel planning. arXiv preprini arXiv:2206. 10680, 2022.
Tom Silver, Rohan Chitnis, Nishanth Kumar, Willie McClinton, Tomás Lozano-Péree Leslie Kael- bling, and Joshua B Tenenbaum. Predicate invention for bilevel planning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 12120-12129, 2023.
Dídac Surís, Sachit Menon, and Carl Vondrick. Vipergpt: Visual inference via python execution for reasoning. Proceedings of IEEE International Conference Computer Vision (ICCV), 2023.
Richard S Sutton, Doina Precup, and Satinder Singh. Between mdps and semi-mdps: A frame- work for temporal abstraction in reinforcement learning. Artificial intelligence, 112(1-2):181-
211, 1999.
Hao Tang, Darren Key, and Kevin Ellis. Worldcoder, a model-based Ilm agent: Building world models by writing code and interacting with the environment. arXiv preprint arXiv:2402. 12275, 2024.
Sylvie Thicbaux, Jörg Hoffmann, and Bernhard Nebel. defense pddl axioms. Artificial Intelli- gence, 168(1-2):38 69, 2005.
Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v. arXiv preprint arXiv:2300111411, 2023.
12

---

Published as a conference paper at ICLR 2025
CONTENTS
Python API for NSPs
14
Additional Details about the Online Invention Algorithm
14
B.1
Predicate Interpretation
14
B.2
Learning HLAs by extending the cluster and intersect algorithm
14
B.3
Clasification-Accrraey- Based Predicate Sets Score Function
16
B.4
Prompting for predicates
16
B.5
Limitations and Failure Cases
17
Additional Experimental Results
17
Example Online I Learning Trajectory
17
Learned Abstractions
18
Further Planning Statistics
25
Additional Environment Details
25
13

---

Published as a conference paper at ICLR 2025
A PYTHON API FOR NSPs
We provide the following Python API on for writing primitive NSPs; get_object (t: Type) get (o: Object, f: str) retrieves the feature with name for object We also have crop_to_objects (os: [equence([b]ecc)] include just the specified list of ob-
returns all objects in the state of type t. get
for cropping the state observation image jects to reduce the complexity for downstream visual reasoning. Finally, there is eval luaauaee__aa suate_simple_.. eeplleasseerttioo (a: str, i: Image) for evaluating the natural language asser- tion in the context of image using VLM.
B ADDITIONAL DETAILS ABOUT THE ONLINE INVENTION ALGORITHM
B. PREDICATE INTERPRETATION
We provide example prompt used to interpret the truth value of the ground atom Direct yon (block5, block6) in the state with cropped observation shown Figure 5 The highlighted text illustrates how condition previous action, previous observation, previous truth value, and object IDs to improve the predicate evaluation accuracy.
Evaluate the truth value of the following assertions in the current state as depicted by the image labeled with 'uurr. state'. For context, the state right after the robot has successfully executed the action Pick(robotI block5:block) The state before executing the action is depicted by the image labeled with prevv. state". Please carefully examine the images depicting the prevv. state' and 'curr. state' before making judgment. The assertions evaluate are:
1. block5 is directly on top of blockó (which was False before the successful execution of the previous action)
prev. state
6
curr. state
Figure 5: Example cropped current (right) and previous (left) observations used for interpreting
ground predicates.
B.2 LEARNING HIAS BY EXTENDING THR cluster and intersect ALGORITHM
We aim to learn high-level actions Q, which define an abstract transition model in the learned predicate space, from interactions with the environment. These interactions sist of executing high-level plans, which are sequences of (grounded) HLAs w,....,,, (.ee HLAs applied to conerete objects). Our learned abstract transition model should both fit the transition dataset while being optimistic for efficient exploration (Tang et al. 2024]. Recalling the definitions from sec. 2 given the current transition dataset, D {..., (x)()) (1)) (k) )..., (zx(1), (4"), -FAIL,,... we first transform it into the learned ab- stract state space, Dv (s(1), n (k) (K) )...,, (s(K') [,(())((()))))))))) (k'), FAII.),.... where
14

---

small Published as a conference paper at ICLR 2025
ABSTRACT (z). We aim to learn high-level actions, Ω2 such that for all high-level actions Oo on objects O
v(s(R), (k) 9% E De, 3a E 26",
w. P.PRE C 5 (k))
(N) ((h) W.EFF A A - -s = W.FFF",
((s(k), x (8) (k) E De, Vu 2o
PRP [ (b())
(K) (k) EFF ^g - -8(h) -..FFF
PRE C s(k) ((s(k), (1), FAIL) E Dv, Bu 1o
where Mo tw Ne:wERA A Aw.T (h))
(3)
while minimizing the syntactic complexity of the HLA, w. PRE +
w EFF W. EFF
To find the high-level actions satisfying this objective, we first split the dataset according the skills, as each high-level action is only associated with one skill, D {d : E Dy A d. = m.). We then split each skill into one or multiple high-level actions by unifying the effects D following the cluster and intersect operator learner (Chitnis et al.) (022). This compensates for the fact that skill can have different effects in different situations, by first partitioning the transition datasets into high-level actions,
Dij {dd De d. W.F A d.s dhglk) EFF* A d.s((k) d.s EFF
where w = w(01, 12,0,,0,..... .). for all E O}.
Each partition associates a high-level action with the skill w. d.t,, AA Dig. while the postcon- ditions of the high-level action (w Err and E.FFF ) are also learned, by unifying and lifting the effects of data in Di. See Chitnis et al. (2022) for more details. For the preconditions, WRPRE, we learn them by maximizing
J (w.PRE)
1
Σ dED
W.PRE S dol(t)
Σ dE DE(Da*r-pp)
I [w..PRE & d.allt) +a W.PPR/
+
(4)
This ensures that all data in the partition is modeled by the associated high-level action, w. specifies that the skill w.r is applicable states g(k) w. PRE (k)) This high-level action also models all other data in the transition dataset, specifying that its precondition is not satisfied if skill not applicable on a state, (s(k), W.m, FAIL) E Di. or if a skill has different effects when applied the state, (s(k) w.m, ((1)) E Dy. (s(k), W. (k))) Di We the parameter number, which softly penalizes syntactically complex preconditions.
Compared with the cluster and intersect operator learner (Chitnis al. 2022), which simply inter- secting over feasible states to build preconditions for each high-level action, our method optimisti- cally enlarges the set of feasible states for each high-level actions using the minimum complexity objective, while still retaining the abilities to distinguish infeasible states. The optimistic objective critical for predicate invention by interactions where optimal demonstration trajectories are not available. Using the intersection method, the agent will only consider the feasible states in the cur- rently curated dataset as feasible and never try the skill in other states that are potentially feasible as well. Planners usually fail to find plans with such restrictive world models, resulting in inefficient random exploration and poor test-time performance.
The restricted preconditions are less generalizable as well. For example, for agents learning making coffee environments with one cup, the agent will find successful trajectories such as PutKettleIn- CoffeeMachine, MakeCoffee, and PourCoffeeInCupp Using the intersection method, the agent sets the preconditions of PourCoffceInCup as KettlelnMachine and KettleHasCoffee as both of them are always among feasible states of the PourCoffeelnnCup action, even though only KettleHasCoffee
15

---

Published as a conference paper at ICLR 2025
is needed. The more restricted preconditions are problematic when generalizing to environments with more than one cups. The agent keeps putting the kettle back to the machine before pouring the coffee for another cup, the learned PourCoffeclnCup action has KettlelnMachine its precondition. The agent eventually fails to solve the problem as the number of cups increases due to the almost doubled length of feasible plans in the more restricted abstract world model. Our method finds the correct precondition as KettleHasCoffee with the optimistic objective. We prefer KettleHasCoffee over KettleInMachine it fails distinguish infeasible states for the Pour skill with different effects, PourNothingInCup.
part of
In terms of time complexity, cluster and intersect linear in the number of successful transitions in D and the number of predicates V, where the additional greedy best-first search (GBFS) that we do introduces exponential complexity with respect to the number of predicates. To balance computa- tional efficiency and performance, we use cluster and infersect in the inner loop of predicate section and then apply our method to the selected predicates (which is usually less than a dozen). Addi- tionally, local hill climbing can be used as an alternative to GBFS to further improve computational efficiency.
B.3 CLASSIFICATION-ACCURRE BASED PREDICATE SETS SCORE FUNCTION
When no satisficing plan is found in early iterations of predicate invention (c.g., in Coffee), the objective from [ilver et al. (2023) is inapplicable. This issue is particularly prominent when the space of possible plans is large (i.e., when there many potential actions each step and achieving goals requires long-horizon plans). To address this, we introduce a predicate score function that does rely on satisficing plans. We propose alternative objective based on classification accuracy, in the same flavour as the score function defined earlier for operator preconditions.
Formally, given De
)).... f..., (R), )( ..... (s(K", x (k'), FAIL), ... where
ABSTRACT (x) as above, we denote the collection of all success transitions and failed tuples as Di {(s(1) (1), (())))} and D {(s(k) m((,, FAIL)) respectively. The the predicate set score function is
J(
1
Σ
1 (3w..T W...E Cs)
D
(2(1), (1), ( (t JEDt
Σ
1 ( (Aw.T(h)
T...PR c²) - + +a. 1v.l.
(5)
(a00) (T00) FAIL)ED
Intuitively, this objective selects for the minimal set of predicates v such that the HLAs learned from these predicates, 2y. avoid attempting to execute a skill states where it has previously failed while ensuring that the HLAs enable the skill to be executed in states where it has previously succeeded.
B.4 PROMPTING FOR PREDICATES
Strategy #1 (Discrimination). is motivated by one of the primary functions of predicates-have them in the preconditions of operators distinguishing between the positive and negative states so the plans the agent find are feasible. However, we observed that existing VLMs often struggle reliably understand and identify the differences between positive and negative states, especially when dealing with scene images that deviate significantly from those seen during training. This limitation motivates our second strategy.
Strategy #2 (Transition Modeling). With the observation that predicates present in an action's preconditions often also appear some actions' effects. We prompt the VLM to propose predicates that describe these effects based on the positive transition segments it collects. This task is usually easier for VLMs because involves identifying the properties or relationships that have changed from the start state to the end state, given the information that action with a natural language name (such as pick) has been successfully executed. However, this strategy alone is not exhaustive. Certain predicates may exist solely within preconditions but not effects (e.g., object's 's material that remains unchanged). Therefore, this method complements S1 and is used alternately with during the invention iterations.
16

---

Published as a conference paper at ICLR 2025
Strategy #3 (Unconditional Generation). prompts VLMs to propose derivations based isting predicates. These derivations can incorporate a variety of logical operations, such as negation, universal quantification (c.g., defining Clear (x) based on On (x, y) ). transitive closure, and disjunction (e.g.. defining OnPlate. p) based on Direct I (x, y) and Di Direct yOnp1 atte (x, ). This approach helps create derived predicates, such OnPla l at for Balanced (fig. D which is unlikely to be proposed by the first two strategies but are essential for correctly implementing complex predicates like Bal anced. As a result, this S3 is used every invention iteration before either S1 or S2 is executed.
For each predicate proposal strategy, we propose a three-step method guide the VLMs: 1) Ask the VLM to propose predicates by providing a predicate name, a list of predicate types drawn from A, and a natural language description of the assertion the predicate corresponds to. 2) Synthesize the predicates classifiers using the syntax and API we provide for NSPs 3) Identify any potential derived predicates and prompt a language model to transform them into the specified function signature for derived NSPs. Given the challenges in S1, we add an additional step 0 just for this strategy. We query the VLM to propose properties or relations among objects in natural language, which are then formalized into predicates in Step 1.
B.5 LIMITATIONS AND FAILURE CASES
A primary limitation of the system is the accuracy and reliability of the VLM in evaluating NSPs.
In some cases, the system can recover from imperfect predicate evaluation accuracies. This is be- cause noisy predicates are not selected during the predicate selection process, and variations of the predicates, with slightly different natural language descriptions, can be proposed later invention iterations. These variations may achieve higher scores, making them more likely to be selected.
In other cases, they never recover. For instance, in the Cover Heavy domain, our initial plan was assign common materials, such wood and metal, blocks distinguish between light and heavy objects. While the predicate proposal VLM successfully suggested appropriate predicates (e.g.. IsWood ( (bblock) and IsMetal (?block) the predicate evaluation VLM was unable to interpret these predicates with sufficient accuracy and consistency to build a useful world model. The issue persisted even after switching to white and black blocks to represent heavy and light blocks and was ultimately resolved by using green and red blocks instead. Similarly, in the Coffee domain, the predicate Is JugFi iefeeddd (? jug) is an essential precondition for the pour HLA. However, the VLM could not interpret this predicate accurately enough, necessitating that we treat it as predefined predicate.
Potential solutions include: 1) integrating proprioception more effectively into the system; 2) devel- oping ways to accurately assign belief scores over the truth values (c.g., use (Jjug 0.9" to denote "I believe the jug is filled with coffee with probability 0.9"); Or designing embeddings for ground predicates and observations, and determining the truth values of ground predicates by comparing distances between the corresponding embeddings.
At the same time, we expect improved accuracy in real-world scenarios compared to simulated domains with poor graphics quality, as there should be less distribution shift relative to the VLMs' training data, and the VLMs have demonstrated very strong performance on simple visual question- answering tasks with natural images Yang al. 2023
C ADDITIONAL EXPERIMENTAL RESULTS
C. EXAMPLE ONLINE LEARNING TRAJECTORY
Figure 6 shows an example of a predicate invention curve in the Coffee environment. Learning begins with 800 failed plans (i.e., unable to solve any tasks) and concludes after 8 iterations when the number of failed plans reaches zero. In total, 9 predicates are selected from 46 candidates. In the end, it selects 9 predicates among 46 candidates.
17

---

Published as a conference paper at ICLR 2025
800
CupFilled
JugFilled
CupFilled
# of Failed Plans
600
FingerOpen
Holding
FingersOpen
JugFilled
CupFilled
JugFilled
Not-JugFilled
JugUpright
FingersOpen
Holding
JuginCotfseMachine
400
NOT-RobotNeerJJu
JugUpright
NOT-JugUpright
Holding
NOT-NOT-
NOT-JugUpright
RobotNearJug
NOT-JuginCoffeeMMshhhn
NOT-JugFilled
200
NOT-JugFilled
1
2
3
5
7
# of Online Learning Iterations
Figure 6: An example online predicate invention trajectory. The bubbles show the predicates being selected among all the candidates it has that iteration.
C.2 LEARNED ABSTRACTIONS
We show examples of learned predicates and operators here.
C.2.1 COVER
"'pythen
def _GripperOpen_ @a0tppen____________ holdd cSs__hodPds_sde (state: Rawstate, objects: Sequence []bject -> bool: zobot, ebjects return get (robot, "fingers") 0.5
"Gripperopen"
param_Lypes: Sequence [Type] - [eobot__yped
GripperOpen NSPredicate (name, paran_types, Gesipee:Ope__MS__holll
** ''python
12 def _alddnng NSP bolda RawState, objecto: Sequence (object)) >> bcol: rosot, block objects
14
If the gripper is open, the zebot cannot be holding anything state.get (robet, "fingers") return False
16
17
the image focus the robot and block iianttio__inage e tttatecero_tt_oojjee robot, block)) robot_name robot.id blme = block.id_name
19
21
returr htteee/wwelaaee_imme_eeeeeeeeee
23
("ooohot_name) 15 holding [b] Balook_name?? attention_image
24
names str "Rolding" 26
Lypest Sequence [Type] LLoobo__yype, _looc__tppe Holding = NSPredicate (name, paran_types, _oolding_MSP NSP MSP_hheee_
23
.*
18

---

Published as a conference paper at ICLR 2025
NSRT-OpC:
Parameters: [2x0:block, 2x1 :robet]
Preconditions ons: [GripperOpen (2x1
:ebbot)] rebb:)
Add Effects: [Holding robot ?x0:block)
Delete Effects: [GripperOpen (Px1:
Ignore Effects: I]
Option Spec: Pick (?x0:block)
NSRT-Op1:
Parameters: [?x0:block, ?x1:robet, ?x2:target]
Preconditions: [Holding (Exl:robet, ?x0:block)]
Add Effects: [Covers (2x0:block, ?x2:target), GripperOpen (?x1 robot)
Delete Effects: [Holding (exl:robet, ?x0:block)]
Ignore Effecta: []
Option Spec: Place :lock,, ?x2itarget)
C.2.2 BLOCKS
Gripping * ''python
def _er___ipinn_gee [SP_heldd RawStale, jectes Sequence (Ob ject]) bool: "#npeternine 15 the scbot in objects is gripping the block in objects in the image. o*e rooot, block objects rogot_name robot. block_name namee
i If the robot's fingers are open, it can't be gripping anything. if state.get (robct, "Eingers") > 0:
11
Falae
13
smallest bounding actention image state.crep ( roboo, black]) returr state.eveluate.
Crop scese image
include both objects.
15
17
["[robot_nama 18 gripping (block_name). attention_imagen
"Gripping"
20
Lypes_ Sequence [Type] - [_eobo__Lype blook_Lype]
Gripping NSPredicate (name, paran_types, _eripping
22
24
Clear
25
'pythor.
Derine classities function
27
def _ClesrCPP holdd (atoms: Set (Groundåt objects: Sequence [Object]]
bool:
"*npetermine
block top of the given block. ena
29
block, objects
31
i Check if any block is on top of the given à ook
in if predicaee -- On and atom.cbjects/1. -- block:
34
return False
36
return True
38
Define the predicate name here
39
name: str "Clear"
41
# A list of object-type variables for the predicate, using the ones defined in the environment param_types: Sequence [Typa] [_blcck_type]
43
44
Instantiate the predicate
45
Clear ConceptPredicate (name, param_types, _lear__P_holal
46
48
imptyGripper
.* ''yyhho
50
det _npptyGripper_Dee _op__hodddaaapppoooooooooaaaaaaa (state: RawState, objecta: Sequence [[b]ec -> bools .#npetecnine the gripper of robot in objects is empty in the scene image, "xr robot, objects
51
33
# If the robot's fingers are clesed, it can't be empty.
if state.get (xobet, "fingers")
55
return False True
19

---

Published as a conference paper at ICLR 2025
name: ser "EmptyGri prrpptrrrippprr" ippr""
param paran_types: types: Sequence [Type] [_robo__type type] EmptyGripper NG2 redicate (name, pasemltypes, _nppty@ripper__eR_.o.
61
NSKT-Op6:
Parameters: [2x0:block, bxl:block, ?x2:robot]
Preconditions IClear (2x1:block), EmptyGripper (P22:coctt))
On (2x1:block, ?x0:block)]
Add Effects: [Gripping rob::rttt,, ?x1:block)
Delete Effects: [EmptyGripper (?x2:robot), On (?x1:block, ?x0:block)]
Ignore Effects: []
Option Spec: Pick roxo: ?xl:block)
NSRT-Opl:
Parameters: [?x0:block, ?x1:robet]]
Preconditions: [Gripping (2x1: rrbbet ?x0:block)]
Add Effects: [EmptyGripper (2x1: OnTable :block)
Delete Effects: [Gripping :obb:r ?x0:block)
Ignore Effects: II
Option Spec: PutOnTable (2xl (2xl
NSRT-Op2:
Parameters: [?x0:block, robot)
Preconditions: [Clear (3x0:block), EmptyGripper
OnTable (?x0 [2x0:block)]
Add Effects: [Gripping (2xl: ?x0:block)]
Delete Effects: [EmptyGripper (exl:robot), OnTable (2x0:block)
Ignore Effects: []
Option Spec: Pick (?x1 ?x0:block)
NSRT-Op3:
Parameters: [2x0:block, ?xl:block, ?x2:robot]]
Preconditions! [Clear (?x0:block), Gripping (?x2: ?x1 :block)
Add effects: [#mptySripper (2x2: On (?x1 :block, ?x0:block
Delete Effects: [Gripping (2x2:robot, ?x1:block)
Ignore Effects:
Option Spec: Stack (?x2 rebot, ?x0 :blokk)
C.2.3 COFFEE
RabocHoldingJJug
JugTilled
">python
_JugTilted__0p (state: (state: RawBLale, jects: Sequeace [[bject]) bool: r#nDetecaine 15 the jug is rotated by non-zers zero angle. jug, objects
Adsuning a rotation value of 0 means upright, other value implisa rotation reture aba tstate.get Jjag, "rot")) 0.1
11 names str "TugTilted"
pacam_lypes: Sequence [Type] Lype_
13
JugTilted RSPredicate iname, paran_types, _
15
JugUpcight
18 * "yyyhhen
SugInMach.ine
20
def _wwggnnacchnnn.ue
**njug (ttprissn_____oooddeedeeeeeeeeee (stazee: RawState, objects: Sequence (Ubfec=]) >> bool;
placed inside coffee maching "y.*ns
*na
22
jug, machine objects
23
the jug is being held, it cannot be in the machine. if _wobottoldinggues holds
ettate.get_objeet (robot_type) [01, jugl):
25
return False
27
Crop the image to focus on the jug and the coffee machine. actention_ Image crate.crop_ta_abfeee jjg, machine])
20

---

Published as a conference paper at ICLR 2025
jug_name jug.id_name name
30
machine_nnmm bachine.id_name
returr evate.evallate.e.e sinpeesssssee__eeeeeeee simple_asaertioon
32
[amjagnaame placed inside {machine_name)... attention_image
33
35
name: str "GugTnMachine"
types: sequence [Type] [_jeg_type, _machine_type
37
Tug ine NSPredicate (name, param_typea, _JugTMMaching ne_ BSP____SP____ 2o____SP_dllaaaph dal
39
40
GripperOpen
21

---

Published as a conference paper at ICLR 2025
NSRT-OpC:
Parameters: [?x0: jug, :xl:robot:
Preconditi ons: [Gr (GripperOpen (2x1: robot), JugUpright (?x0 jug) ]
Add Effects: [RobotHoldingJug (?x1
?x0: jug)
Delete Effects: [GripperOpen :robbtt
Ignore Effects: II
Option Spec: PickJug : ?x0: jug)
NSRT-Opl:
Parameters: 10x0: maccheneaaaeneeeeeeeeeea ?x1: jug, ?x2:robot]
Preconditions: [RobotHoldingJug (?x2 robott, ?x1:jug)
Add Effects: [GripperOpen (?x2:rebet),
Ing (7x1: jug, #X0:coffee_machine) (P
Delete Effects:
?x1:jug)
Ignore Effects: []
Option Spec: alace.uggnMacchnee (?x2:robot, ?xl:jug, MM0:coffee_machine
NSRT-Op2:
Parameters: [ox0:coffee_machin machiin,,, ?xl: ?x2 robot]
Preconditions: [JugInMachine (?x1: jug, ?x0: wwwfoff_efchhnnneeee
Add Effects: [JugFilled (2x1:jug)
Delete Effects: []
Ignore Effects: []
Option Spec: TurnMachineOn (?x2 rebot, ?x0: coofce_emmcchine
NSRT-Op3:
Farameters: Set::ooffe_maccinne ?x1: jug, robott
Preconditions: [JugInMachine : 7x0: :x0::eoffe_mmee coffee_machiiee)
Add Effects: [RobotfolllingJug rexo: ?xl:jug)
Delete effects: (GripperOpen (2x2:robat), Jug IMgacchhineeee (?x1: "x0:coffee_maching
Ignore iffects: []
Option Spec: PickJug robo: jug)
NSRT-Op4:
Parameters: [?x0:cup, ?xl:jug, rx2oro
Preconditions: [Jugrilled ("xl:ju
RobotHolcingJug robot ?xl:jug)
Add Effects: (CupFilled(xxxxcup))
Delete Effects: [JugFilled Robot Hologggg robot ?x1:jug)
JugUpright (jgl:jug)
Ignore Effects: []
Option Spec: Pour (7x2:robor jugg ?x0:cup)
NSRT-Op5:
Parameters: [7x0:jug; ?x1:
Preconditions: [GripperOpen (2x1 rcbott)
Add Effects: [JugUpright (2x0: jug))
Delete Effects: []
Ignore Effects: []
Option Spec: Twist (?x1: robot, ?x0:jug)
22

---

Published as a conference paper at ICLR 2025
NSRT-Op6:
Parameters: :ex::coffee_mach ne, ?x1: jugg ?x2: robot]
Preconditions ons: [JugInMach] InMachie (?x1 jjug, :x0:coffi weeeefehineneeeeeee
Add Effects:
Delete Effects: [JugInMachine j 7x0:coffee )
Ignore Effects: II
Option Spec: TurnMachineOn (2x2: robot, rx0:coffee_maching
NSRT-Op7:
Parameters: 1?x0:cup, ?xl:jug, rx2:robot]
Preconditions: (Px:
?x1:jug)
Add Effects: [CupFilled (2x0:cup), (ug]iltee
Delete Effects: [JugFilled :exl:jug) RobotHolcingJug robot :xl:jug)
Ignore Effects: []
Option Spec: Pour (?x2: ?x1:jug, ?x0:cup)
NSRT-Op8:
Parameters: [ux0:cup, ?xl:jug, ?x2:robot]
Preconditions: [JugFilleddxx]: jug))
RobotHoldingJug robotr ?x1:jug)]
Add Effects: [Cuprilledd?x0
[ugTilted
Delete Effects: []
Ignore Effects: []
Option Spec: Pour robot, 2x1: ?x1:jug, ?x0:
NSRT-Op9:
Farameters: cux0:cup, ?x1: jug, ?x2: roborobot]
Freconditions: [JugFilled?>x] jug), RobotHoldingJug (?x2 : robot, ?x1:jug)]
Add effects: [CupFilled (2x0:cup), JugTilted (2x1:jug)
Delete iffects: bobbottoldingugg (?x2: robot, Px1:jug)
Ignore Effects: II
Option Spec: Pour (?x2 robot, ?xl jug, ?x0:cup)
C.2.4
COVER HEAVY
EmptyHand
Holding
IsBlack
" "'yython
der TTTRlack NSP wsp_holda State, objecta: Sequence (object)) -> hool:
block, objects
block_id black.1d_namg
actention_image state. [book]])
zeturn seate./vslaasee_simme_eeeeeee (blockkkkjdd is bleck.", sctention_imege)
"IsBlack"
12
param_types [bbloc_ttpp]
IsBlack red (nane, paran_types, _IsBlack_oos
14
23

---

Published as a conference paper at ICLR 2025
NSRT-Opl:
Parameters: [?x0:block, 2x1 2xl: robot, ?x2:target]
Preconditions ons: [Halding : ?x0:block)]
Add Effects: [Covers (Px0:llkkk,,, ?x2itarget), EmptyHand (?x1 1: rrbbot)
Delete Effects: [Holding robot,, ?x0:block)
Ignore Effects: II
Option Spec: Place (?x0:block, ?x2:target)
NSRT-OpC:
Parameters: [?x0:block, ?x1:robet]
Preconditions: [IsBlack (2x0:block), EmptyRand (?xl :robct)
Add Effects: (fx1:
[xocklock)]
Delete Effects: [EmptyHand (2xl:rebat.)
Ignore Effecta: []
Option Spec: Pick (2x0:block)
C.2.5 BALANCE
def _nnnnnnteeeeec helldde____h (atems: Set (eronddhhom
objects: Sequence (Cneeee[Cjeeeee [ebjeet] -> bool:
x, y objects
in
If atsm.pcedicate -- DirectlyOnPlate and) atom.ongects [x, yl
return True
other blocks [a.objests for a in atoms
s.predicate Directlyun er)
12
a.predicate Direct lynn
14
for otock_block in blocks:
holds False
for atem in stoms:
17
if atem.predicete DirectlyUn and! atom.ohjacta
othar_block):
19
holda1 True
break
21
LE holdsl and Ao (cther_block, yl):
return True
23
return False
24
"OnPlate"
26 param_types: Sequence [Type] - [_bloo__typ Lype, _plate_type]
OnPlate Concept2redicate (name, param_types, _One _Onilate_CP_aat
28
31
oockhistrrii butedRvenly Ca_hcca_lllll objects: Sequence [Object]] bool: platel, plete2 objects
GroundAAtom),
33
35
if platei == plate2:
return countl 0 in if alsm.predicate -- OnPlate:
count2
40
atom.objects(1] platel: count += elif ca place2: count2 returr count1 == count2
42
45
47
name: str "gJoctsüistribbteeeeeeiii
types: Sequence [Tyre] [plate_type, _plate_typel
49
BlocksDistri butedRuenly pate (name, paran_typas,
24

---

Published as a conference paper at ICLR 2025
NSRT-Unstack:
Parameters: [?black:block, octherbl ?robot: rebett rabet]
Preconditions [Clear (2block:bllccc Directl (2b1 ock blokk, Fotherblock:block)
(aapper :robott)rrooo]
Add Effects: IClear (Fstissrlock:bboock) Holding (bokkc Delete Effects: [Clear (?block:block), DirectlyOn (?block:bleck Focherblock:block)
Ignore Effects: []
Option Spec: Pick (Trobot: ?block:block)
NSRT-Op3:
Parameters: [?block:block, :otherbllock:block, ?robot: :cbbct]
Preconditions: [Clear ( (2otherbllcck:bblckk Holding rbllckkbblckk
Add Effects: [Clear (bblock:block), Directlyõn ( (bblock.bbocc, fotherblock:block)) aripperopen
Delete Effects: IClear (Fotherblock::bloch)) Holding (2block:block)
Ignore Effects: I]
Option Spec: Stack prrbbotrrbbet. Fatherblock:block
NSKT-Op2:
Parameters: [?x0:machine, ?xlttlate, ?x2:plate, ?x3:rcbet]
Preconditions:
(ttttrrbuttedEeeelyyyyyaaappa ?xl:plate)] Add Effects: [MachineOn (2x0 machhne))
Delete Effects: []
Ignore Effects: []
Option Spec: TurnMachineon (?x3: ?x1:plate, ?x2:plate)
NSRT-Op4:
Parameters: [?block:block, ?robet: ?plate:plate]
Preconditions [ClearPlate (?plate:plate) Holding (2block:block)
Add Effects: [Clear (Pblocckbb:: DirectlyOnPlate bolkkkk ?plate:plate), GripperOpen (2 (Probot:
Delete Effects: IClearF late (?plate:plate), Holding (2block:block)
Ignore Effects: II
Option Spec: PutOnPlate (?robot rebot, ?plate:plate) NSKI=Picckrromiaalle:
Parameters: [?block:block, rrobet: ?platepplate]
Preconditions: [Clear (bblockbblokkk ?plate:plate), GripperOpen (2ooootttot)
DirectlyOnPies (2block:block)
Add Effects: (Reldding(2blooc:bblol))
Delete Effects: [Clear (bolockkbkk))) DirectlyOnPlatt (?block: block, ?plate:plate), GripperOpen (2robct:
Ignore Effects: []
Option Spec: Pick (?robot: :oboot, bblock:
C.3 FURTHER PLANNING STATISTICS
The average plamning node expanded and wall-time statistics for our approach, alongside other plan- ning approaches, are summarized in Table
In the Blocks and Balance domains, our use of derived predicates is not out-of-box compatible with relaxed planning heuristics, such LM-eut, which typically employ through Pyperplan. As result, we resorted to simpler goal-count heuristic, which estimates the distance to the goal by counting the number of unsatisfied goals. This heuristic is less informed than LM-cut, leading to significantly larger node expansions and longer planning times in these domains than expected. In future work, we aim develop a version of LM-cut that is compatible with derived NSPs.
D ADDITIONAL ENVIRONMENT DETAILS
Cover: This environment has goal predicate (Covers (2x:block, ?y:target) }. The ini-
tial operators
25

---

Published as a conference paper at ICLR 2025
Ours
Oracle
Sym. pred.
Environment
Succ
Node
Time
Succ
Node
Time
Succ
Node
Time
Cover
100.0
9.4
0.142
100.0
8.4
0.129
100.0
26.9
0.151
Blocks
96.0
1117675
254.621
94.0
550630
101.737
7.2
121.4
4.279
Cover Heavy
97.0
7.9
0.057
100.0
5,4
0.060
46.0
5.7
0.061
Coffee
65.3
40.3
0.969
99.3
19.3
0.652
68.0
199.4
3,270
Balance
100.0
26.3
0.856
100.0
30.6
0.585
20.0
12.2
0.125
Ours
Ablate op.
No invent
Environment
Succ
Node
Time
Succ
Node
Time
Succ
Node
Time
Cover
100.0
9.4
0.142
100.0
7.0
0.148
68.0
28.1
0.113
Blocks
96.0
1117675
254.621
12.0
24.8
0.222
1.3
321.0
0.224
Cover Heavy
97.0
0.057
46.0
5.7
0.128
36.7
29.5
0.099
Coffee
65.3
40.3
0.969
65.3
29.6
2.441
0.0
Balance
100.0
26.3
0.856
100.0
28.0
1.180
25.3
13.5
0.204
Table 1: Further planning statistics.
NSRT-Pick:
Farametors: [Pblock:block
Preconditions: []
Add Effects: []
Delete Effects: []
Ignore Effects:
Option Spec: (7bbcckkbk))
NSRT-Place:
Parameters: [?block:block 2target:target)
Preconcitions: II
Add Effects: ICovers (bblock:bboo,k ?target:target
Delete Eftects: I|
Ignore Effects: []
Option Spec: Place (?block:block itarget:target
Blocks. This environment has goal predicates: OnTable (2x:block) } and initial operators
[on (?x:block, ?y:block),
26

---

Published as a conference paper at ICLR 2025
NSRT-PickFrrom'abll::
Parameters: [7black:bloss) 2robet:rabat)
Preconditions: []
Add Effects: II
Delete Effects: [OnTable (bblock:block)
Ignore Effects: I]
Option Spec: Pick :rrobot:robot., Pblock:block)
MSKI-FutOnTable:
Parameters:
l7block:block, ?robot:
Freconditions:
Add Effects: [OnTable (#block:block)
Delete Effects: []
Ignore Effects: []
Option Spec: PutOnTable (robbtt:robot)
NSRT-Stack:
Farameters: [?block:block, Potherblock:block.
probot:robor
Preconditions: []
Add Effects: [On (?block:block,
Potherblock:block)
Delete Effects: []
Ignore Effects: []
Option Spec: Stack (?robet: rebott,
totherblock:blass
NSRI-Unstack:
Farameters: [?block:block, Potherblock:block,
Probot:robot.
Freconditions: []
Add effects: []
Delete 6ffects: [On (bblock:bl
Pothezbblock:bloekk]
Ignore Effects: []
Option Spec: Pick (?robot rrbott, ?block:block)
Coffee. This environment has goal predicates: {CupFilled ( (rup:cupp) }. We include the predicate JugFilles (? ju:: jug) the initial set of predicates because was very challenging to have VLM to determine this especially with the graphics in the simulator. It has initial operators:
27

---

Published as a conference paper at ICLR 2025
NSRT=Pickiing>ro/lablle
Parameters: Prrobotrrobot, ?jug:jug]
Preconditions: []
Add Effects: II
Delete Effects:
Ignore Effects: II
Option Spec: PickJug (?robot: robot, ?jug: jug)
Parameters: 1 ?jug: jug,
?machine:coffes machinel
Freconditions: []
Add Effects: []
Delete Bffects: []
Ignore Effects: []
Option Spec: PlaceJugInMachiine (?robot: robot,
?jug: jug, mmachins:toffee_mcchiie
MSKT-PourFromtowhere:
Parameters: rroobo jug,
?cupicup]
Preconditions: []
Add Effects: [CupFilled (?cup:cup)
Delete Effects: []
Ignore Effects: []
on Spec: Pour (Probot:robe) bjug:jug;
?cup:cup),
ISRI-TunmMachineon:
Farameters: [?robot: : robot, Jug,
seaccine:coofee_meee
Preconditions: []
Add Rffects: [Jugrilles (Zjug:jug)
Delete Effects: |]
Ignore Effects: []
Option Spec: TurnMachineon (?robot: robot,, "machine:coffee_mahhhe
NSRT-Iwist:
Parameters: 1Probot:robet ?jugrjug)
Preconditions: II
Add Effects: []
Delete Effects: []
Ignore Bffects: []
Option Spec: Twist (?robot trbbot, ?jug:jug
Cover Heavy.
This has the same set of goal predicates and operators Cover.
Balance. This has the goal predicate: [MachineOn (?x (?x:machine)] Here we con- sider continual learning setting where the agent is initialized with the abstractions com- monly found Blocks. They are [Clear (2x:block), ClearPlate (?x:plate), DirectlyOn (?x:block, ?y:block), , DirectlyonPiate (2x:block, ?y:plate), GripperOpen r?x:robot), Holding (2x:block)). The initial set of operators is:
28

---

Published as a conference paper at ICLR 2025
N
Parameters: [?block:block, brobet:rabar
?plate:plate]
Preconditions: [Clear (bblock:block),
DirectLyOnFlate (bblokkbbbbb, ?plate:platel, (Prrbottot((ccooo:rrobbtt)))))
Add Effects: [Holding (?block:block)]
Delete Effects: [Clear (zblock:block),
DirectlyonFlate (fblock:block, ?plate:plate),
GripperOpen (?robot:robat))
Ignore Effects: []
Option Spec: Pick (?robot: semoorrroke robot, ?block:block)
NSRT-FutonFlate:
Parameters: [?block:block, ?robot:
?plate:plate]
Preconditions: [ClearPlate (Pplattealt)))l
Holding (2bloo::bbkkkkkkkk
Add Effects: [Clear (bblockbbkk))
DirectLyOnPLate (7blockkbbb,,, ?plate:plate),
pripperppen (Probot:robot)]
nelete 8ffects: [clearPiate [[aaarrlaaadeeplllatspplaaa)))) Holding (?block:block)]
Ignore Effects: []
Option Spec: PutOnPlate (2robot:robot, ?plate:plate),
NSKT-Stack:
Parameters: [?block: :blokk,, batberrlook:blooooo
probot:robot]
Freconditions: [Clear (Rotherblockkblook))
Holding (bblock:block)
Add Effects: [Clear (bblock:block))
(f GripperOpen (Probot: robott]
Delete Effects: [Clear (bolhetbblokkbblkkk)
Holding (2blockbblock)
Ignore Effects: []
Option Spec: stack (?robot: Potherblock:block)
NSRT-Unstack:
Parameters: [bblock:block, Potherblook::block,
Frobot:robot]
Freconditions: (CIaa(((blck:
Dioecciiooothhook...oo.d botherbloock.lock),
Gri (riaperrpp
Add Effects: [clear//ttherrllooktboe))) Holding (?block:bloock)
Delete Effects: [Clear (bblccckbbk)) Directlyon block, bother)llockblook), (EtpperOpen(rrobotteee
Ignore Bffects: []
Option Spec: Pick :rrobot:rbbss ?block:block)
NSRT-TurnMacchineo::
Farameters: [Probot: 2machine:machine,
:plate, ?plato2:platc]
Freconditions: []
Add Effects: [MachineOn (emachine:machine)
Delete Effects: []
Ignore Effects: []
option Spec: TurnMachineon (?robot (Probot robot, ?platel:plate, ?plate2:plate)
29


Reviews:
[{"review_id": "ALllur7rTV", "reviewer": "Reviewer_oEZx", "summary": "This paper introduces Neuro-Symbolic Predicates (NSPs), a first-order abstraction language that combines symbolic and neural knowledge representations for high-level planning in robotics. The authors propose an online algorithm for inventing such predicates and learning abstract world models from interactions with the environment, without relying on demonstrations. The method leverages Vision-Language Models (VLMs) to ground predicates in perceptual data and integrates them into a planning framework that interleaves predicate proposal, validation, and goal-driven exploration. The approach is evaluated across five simulated robotic domains, comparing it with hierarchical reinforcement learning, VLM planning, and symbolic predicate invention methods. The results demonstrate that the proposed method offers better sample efficiency, stronger out-of-distribution generalization, and improved interpretability.", "questions": "1. How would your method scale to real-world environments with more complex visual scenes and possible noise? Have you considered the potential limitation of current VLMs in these settings, and what could be the effect on NSPs' performance?\n\n2. Considering the fact that VLMs are not infallible, how sensitive is your approach to inaccuracies or biases in the VLM outputs? Do you observe cases where VLM errors lead to incorrect predicate invention or planning failures, and how could these be mitigated?\n\n3. Your approach assumes a given fixed set of low-level motor skills. How much can that assumption be relaxed, and could the approach be extended towards learning or adapting these skills jointly with the high-level predicates?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. Novel Approach: This paper proposes a novel approach to effectively integrate symbolic reasoning with neural perceptual representations using NSPs to address the challenge of forming task-specific abstractions in robotics.\n\n2. Online Predicate Invention: The proposed method uses online invention of predicates from environment interactions that does not need demonstrations. This is a valuable property in real-world applications, since collecting demonstrations could be expensive or even infeasible.\n\n3. Integration of VLMs: VLMs are leveraged to enable the agent to handle rich representations of concepts, both visually and logically, grounding predicates in the perceptual data, thus enabling more flexible and stronger abstractions.\n\n4. Strong Empirical Results: Their results show strong empirical results, outperforming other baselines over various domains in terms of sample efficiency and generalization to unseen tasks, and planning efficiency.\n\n5. Interpretability: The model uses NSPs, providing interpretable outputs. This helps in understanding or debugging the behavior of the agent. This is critical in safety-critical applications.", "weaknesses": "1. The reliance on VLMs raises questions about its scalability to more complex or noisy real-world settings in which VLMs might perform poorly, hence affecting the robustness of the method.\n\n2. In their approach it is assumed that there is access beforehand to well-defined motor skills and object-centric state representations that in practice are not always available.\n\n3. Comparisons to more SOTA methods in hierarchical planning or neuro-symbolic reasoning could strengthen empirical evaluations and position the work better within the literature.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "B6QHuaLXrH", "reviewer": "Reviewer_qh55", "summary": "This paper presents a new method to learn grounded symbolic predicates and operators from online interaction data for high-level planning. The core of this method is to generate predicates represented in python functions by prompting an VLM. Such predicate functions can invoke VLMs to process image observations and incorporate logical structure to build predicate hierarchies (i.e., derived predicates). Then the most useful predicates are selected and operators are learned following variants of previous works [1, 2]. In experiments, the method is evaluated on 5 domains with multiple baseline methods including Hierarchical RL and VLM planning, showingcase that the method can recover symbolic planning domains that allows effective planning.\n\n[1] Silver, Tom, et al. \"Predicate invention for bilevel planning.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 37. No. 10. 2023.\n\n[2] Chitnis, Rohan, et al. \"Learning neuro-symbolic relational transition models for bilevel planning. In 2022 IEEE.\" RSJ International Conference on Intelligent Robots and Systems (IROS).", "questions": "See Weakness", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper has sufficient quality and soundness, and I'm satisfied with its approach and evaluations.\n- The paper is well-structured and clearly-presented. It has good coverage of literature.\n- Learning symbolic predicates is foundamental to high-level planning. The paper proposes a flexible approach to learn neural-symbolic predicates grounded on image observations and natural language.\n- The proposed method is comprehensively evaluated via comparison with baselines and ablation studies.", "weaknesses": "- I would suppose the VLM-based predicate functions can get wrong even after applying the techniques described in line 178-189. How will the invented predicates with wrong grounding affect downstream operator learning and planning? \n- I wonder how are the low-level skills realized in experiments for the proposed method and baselines. Are they realized by motion planning? Do you learn low-level skills in the MAPLE baseline?\n- In Figure 4, it seems that the oracle baseline doesn't achieve 100% success rate in Blocks and Coffee domain. Its performance is even slightly worse than the proposed method in Blocks. This seems to be weird as the Blocks domain is the most well-known classical domain that can be solved completely with a few predicates and operators. I wonder why this happens?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EgUspLY2CN", "reviewer": "Reviewer_JR1Z", "summary": "This paper proposes a novel framework for robot planning by integrating NSPs, a method that combines neural and symbolic representations. The authors present an algorithm for online predicate invention, leveraging VLMs to dynamically learn abstract world models through real-time interaction in various robotic environments. The study compares the proposed approach to several HRL and symbolic methods, evaluating performance in terms of sample efficiency, generalization, and interpretability.", "questions": "1. How does the framework ensure a robust and diverse predicate set while maintaining task relevance?\n2. Is there a systematic mechanism for leveraging failures to refine or adjust future planning decisions, rather than simply discarding infeasible options?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.\t**Innovative Approach**: The paper introduces neuro-symbolic predicates to robot planning, using VLMs to invent predicates in a task-relevant manner, a strategy that has clear advantages over static symbolic approaches. The approach also supports perceptual abstraction and logical complexity, enabling the system to better generalize across environments.\n2.\t**Experimental Design**: The authors evaluate their approach across five different simulated robotic domains with various tasks, making the results generalizable. Furthermore, the comparisons with other SOTA, are well-chosen and highlight the advantages of the proposed method in terms of efficiency and adaptability.\n3.\t**Methodology and Algorithmic Design**: The online NSP invention algorithm is well-structured and incrementally builds task-relevant abstractions. By leveraging both VLMs for perceptual understanding and a symbolic component for logical manipulations, this approach improves sample efficiency, crucial in robotic applications where data is limited or expensive to obtain.", "weaknesses": "1.\t**Clarity of Explanation**: Some explanations in the paper are unclear. For example, in the paragraph beginning with line 178, the process of conditioning on the previous action should be explained in more detail. Additional details on how these predicate proposals are validated would improve the comprehensibility of the approach.\n2.\t**Reliance on VLMs and Practicality**: A potential limitation of this approach is its reliance on the accuracy of VLMs, which may be sensitive to changes in the environment. The approach assumes that perceptual cues relevant to task abstractions are identifiable through VLMs. However, variability in VLM performance, particularly with domain shifts or challenging visual inputs, can affect the system's robustness. Although the authors attempt to address this by conditioning predicates on past actions and object labeling, it is unclear how effective these strategies will be in practice, especially under less predictable real-world variations where perceptual noise could affect predicate accuracy.\n3.\t**Time Complexity and Performance**: As an online learning method, computational efficiency is critical, particularly for real-time applications in robotics. While the paper presents experimental results in supplementary materials, these do not convincingly demonstrate that the proposed approach maintains efficiency at scale. Including more detailed time complexity analysis or exploring algorithmic optimizations could address this concern, as current evaluations suggest that scalability in time-sensitive or resource-constrained environments may be limited. Moreover, the supplementary results lack a detailed breakdown, which would clarify how computational resources scale with task complexity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Spotlight)