Learning Efficient Abstract Planning Models that Choose What to Predict
Authors: Nishanth Kumar
Willie McClinton
Rohan Chitnis
Tom Silver
Tomás Lozano-Pérez
Leslie Pack Kaelbling
Venue: CoRL 2023 Poster (2023)
Keywords: Learning for TAMP
Abstraction Learning
Long-horizon Problems

Abstract: An effective approach to solving long-horizon tasks in robotics domains with continuous state and action spaces is bilevel planning, wherein a high-level search over an abstraction of an environment is used to guide low-level decision-making. Recent work has shown how to enable such bilevel planning by learning abstract models in the form of symbolic operators and neural samplers. In this work, we show that existing symbolic operator learning approaches fall short in many robotics domains where a robot's actions tend to cause a large number of irrelevant changes in the abstract state. This is primarily because they attempt to learn operators that exactly predict all observed changes in the abstract state. To overcome this issue, we propose to learn operators that `choose what to predict' by only modelling changes necessary for abstract planning to achieve specified goals. Experimentally, we show that our approach learns operators that lead to efficient planning across 10 different hybrid robotics domains, including 4 from the challenging BEHAVIOR-100 benchmark, while generalizing to novel initial states, goals, and objects.

Learning Efficient Abstract Planning Models that Choose What to Predict
Nishanth Kumar njkkcsail mwl.l.mit.ee. edu
Willie McClinton wbm3@csail mii. edu
Rohan Chitnis cenuchit@meta. com
Tom Silver tslvrecsail semtessail.iit......... mit edu
Tomás Lozano-Pérez t tlp@csail mit edu
Leslie Pack Kaelbling lpk0csail mit edu
MIT CSAIL, Meta AI
Abstract: An effective approach to solving long-horizon tasks in robotics do- mains with continuous state and action spaces is bilevel planning, wherein high- level search over an abstraction of an environment is used to guide low-level decision-making. Recent work has shown how enable such bilevel planning by learning abstract models in the form of symbolic operators and neural plers. In this work, show that existing symbolic operator learning approaches fall short in many robotics domains where robot's actions tend to cause a large number of irrelevant changes in the abstract state. This is primarily because they attempt to learn operators that exactly predict all observed changes in the abstract state. To overcome this issue, we propose to learn operators that 'choose what to predict' by only modelling changes necessary for abstract planning to achieve specified goals. Experimentally, lead to efficient planning across 10 different hybrid robotics domains, including 4 from the challenging BEHAVIOR- 100 benchmark, while generalizing to novel initial states, goals, and objects.
show that our approach learns operators that
Keywords: Learning for TAMP, Abstraction Learning,
Problems
Introduction
Solving long-horizon robotics problems in domains with continuous state and action spaces is tremely challenging, even when the transition function is deterministic and known. One effective strategy is to learn abstractions that capture the essential structure of the domain and then leverage hierarchical planning approaches like task and motion plamning (TAMP) to solve new tasks. A typi- cal approach is to first learn state abstractions the form of symbolic predicates (classifiers on the low-level state, such as InGri InGripper) then learn operator descriptions and samplers in terms of these predicates [1, 2]. The operators describe partial transition model in the abstract space, while the samplers enable the search for realizations of abstract actions in terms of primitive actions. In this paper, focus the problem of learning operator descriptions from very few demonstrations given set of predicates, an accurate low-level transition model, and a set of parameterized con- trollers (such as Pick (x, Y, z)) ) that serve as primitive actions. We hope to leverage search- then-sample bilevel planning [3, 2, 4] using these operators to aggressively generalize highly variable set of problem domain sizes, initial states, and goals in challenging roboties tasks.
A natural objective for the problem of learning good abstract model is prediction accuracy [3, 1], which would be appropriate if we were using the abstract model to make precise predictions. Instead, our objective is to find an abstract model that maximally improves the performance of the bilevel
Equal contribution
7th Conference Robot Learning (CoRL. 2023), Atlanta, USA.

---

Ground Atoma
Ground Atoms (Honc)
Beachab-c (quaa()
Reachilli [oooks)
Feachab (Pock])
Reachible (bookT)
Beachis [quan))
Beachabie (quana)
Beachable (vkdoo4)
Figure I: Example demonstration transition and evaluation task from BEHAVIOR-IOW task. (Left) Visu- alization and high-level states for an example transition where the robot moves from being in range of picking a board game box and book to picking 2 books, 2 board game hoxes and video game box. (Right) Visual- range of picking any objects.
ization of evaluation task where the robot starts
planning algorithm, given the available data. The difference between these objectives is stark in many robotics domains where robot actions can affect many relationships among the robot and objects in the world. Making precise predictions about these state changes might require very fine-gratie fine-grained abstract model with many complex operators. Such model would require a lot of data to learn reliably, be very slow to plan with, and also be unlikely to generalize novel tasks.
For example, consider the Sorting Books task from the BEHAVIOR- benchmark [5]. The goal is to retrieve number of books strewn about a living room and place them on a shelf. A Reachable (?object) predicate given indicate when the robot is close enough to an object pick it up. When the robot moves pick particular object, the varies depending their specific configuration. Figure shows transition where the robot moves to put itself in range of picking up book?, 7, but happens to also be in range of picking up several other items. Optimizing prediction error on this transition would yield a very complex operator that is overfit to this specific situation and thus neither useful for efficient high-level planning nor gener- alizable to new tasks with different object configurations (c.g. the test situation depicted the right panel of Figure 1, where the robot is initially not in reachable range of any objects).
objects that reachable
Bo-MMoveToBook-PPeeee Predictionn Error: Args: TobjA 'objB 7objC TobjD TobjE
2objF 2objG
Preconditions: (and (Reachable ?objA)
Args: 2ohjA
(Reachable 2objB))
Preconditions: 0)
Add Effects: (and (Reachable 7objC)
Add Efffett:: (Reachable YobjA)
(Reachable (eech)(Reachable 2objE)
Delete Effects: : (V?x. ?objA o 2x =>
(Reachable RReachlle ?objG))
(Reachable ?x))
Delete Effects: (and (Reachable ?objA)
(Reachable 2objB))
In this work, we take seriously the objective of learning a set of operators that optimizes overall planning performance. We observe that operators need only model predicate changes necessary for high-level search. Optimizing this objective enables learned operators to yield better generalization and faster planning. Our main contributions are (1) the formulation of an operator learning objective based planning performance, (2) procedure for distinguishing necessary changes within the high-level states of demonstrations, and (3) algorithm that leverages (1) and (2) to learn opera- tors via a hill-climbing search. We test our method on a wide range of complex robotic planning problems and find that our learned operators enable bilevel planning to solve challenging tasks and generalize substantially from a small number of examples.
2 Problem Setting
We aim develop an efficient method for solving TAMP problems in complex domains with long- horizon solutions given structured low-level continuous state and action spaces [4, 1, 2]. We as- sume an 'object-oriented" state space: state e X' is characterized by the continuous proper-

---

ties (e.g. pose, color, material) of a set of objects. Actions, u U, are short-horizon policies with both discrete and continuous parameters, which accomplish a desired change in state (e.g., Pick (block, 0) where a grasp transform). These parameterized controllers can be imple- mented via learning or classical approaches (e.g. motion planning). We opt for the latter in all domains in this paper. Transitions are deterministic and a simulator : X - X predicts the next state given current state and action. The state and action representations, well as the transition function can be acquired by engineering learning [6, 4, 7].
PEEIAAGEBACKCHHANNNNGG(G
u) (,g))
HILLCLIBBINSSEARHHIZ
1 4- (en on < 9
Jua t- 00; Jeum J(Q,, D)
for + n
1, n 2, .,,
2 Q
Sc.
3 while Jewr < Jus do
ABSTRACT(2:) ABSTRACTCE) best - FINDEESTCONSSSENNOOO..
S'' 4- IMPROVECOVERAGGF(. D)
5
if J D) Jene then
31. 8 04++1 O)
St t Dt ; Jam + J(S, D)
if w = Null then
7
S + REDICCCCOMLEXITTC... D) if JJS D) Jenre then
break
wi+1 Shestt
S J + J(S,,
P4t1 U (ac+1 E
Just f Jem i Jour + J(22,
return (w541,...,de Wn), (o%y..., Oin)
11
return
Algorithm Preimage backchaining procedure
Algorithm HUL.CL.MBINGSEARCH Icarns operators that optimize objective J.
(details in ($C)).
Since planning in this low-level space can be expensive and unreliable [8, 2, we pursue search- then-sample bilevel planning strategy in which search in an abstraction is used to guide low-level planning. We assume we are given a set of predicates to define discrete properties of and relations between objects (e.g., On) via classifier function that outputs true or false for tuple of objects in a low-level state. Predicates induce a state abstraction ABSTRACT the set of true ground atoms in (c.g., {HandEmpty (robot), On (b1, b ... }). The low- level state space, action space, and simulator together with the predicates comprise environment. To enable bilevel planning for an environment, we must learn a partial abstract transition model over the predicates in the form of symbolic operators.
S where ABSTRACT(C is
We consider standard Beaming-from-demmone tasks with demonstrations, and must generalize some held-out test tasks. A task T T is characterized by set of objects O, an initial state zo X. and a goal g. The goal is a set of ground atoms and is achieved in if C ABSTRACT(2)). A solution to task is sequence of actions π == (wI,.... Un) that achieve the goal (g C ABSTRACT(I.). and x; f(I;-1, u,) for 1 < < n) from the initial state. Each environment is associated with a task distribution. Our objective is to maximize the likelihood of solving tasks from this distribution within a planning time budget.
setting where we are given a set of training
Operators for Bilevel Planning
Symbolic operators, defined in PDDL [9] support efficient planning, specify transition fune- tion over our state abstraction. Formally, an operator w has arguments 5. preconditions P. add effects E+, delete effects E-, and a controller C. The preconditions and effects are each expres- sions over the arguments that describe conditions under which the operator can be executed (e.g. Reachable (?bookl)) ) and resulting changes to the abstract state (e.g. Holding ( (?bookl)) respectively. The controller a policy (from the environment's action space), parameterized by some of the discrete operator arguments, as well as continuous parameters whose values will be chosen during the sampling phase of bilevel planning- E-, C). Given a ground operator w = (P, C),),, if P C then the successor abstract state s' is (s \ E-) U E+ where is set difference. We use F(s, w) s' to denote this (partial) abstract transition function.
substitution of arguments to objects in-
duces ground operator ل =
3

---

11111 1311
Figure Environments. Visualizations for Screws, Satellites, Painting, Collecting Cans, and Sorting Books.
In previous work on operator learning for bilevel planning [3, 2}, operators are connected to the underlying environment via the following semantics: if F(s, w = s', then there exists some low- level transition (z,, u, z²) where ABSTRACT(Z) s, ABSTRACT(2") s', and C(0) for some A. These semantics embody the "prediction error" view in that ABSTRACT must predict the entire next state (ABSTRACT(22") s'). Towards implementing the alternative "necessary changes" view, will instead only require the abstract state output by F to be subset of the atoms in the next state, that is, ABSTRACT(2') S s'. Thus, we permit our operators to have universally quantified single- predicate delete effects (e.g.. V?v. Reachable e (?v)). Intuitively, F now responsible only for predicting abstract subgoals to guide low-level planning. rather than predicting entire successor abstract states. Under these new semantics, F encodes an abstract transition model where the output atoms s' represent the set of possible abstract states where these atoms hold.
Given operators, we can solve new tasks via searech-heen-samaee bilevel planning (decribed in detail in appendix ($A) and previous work [10, 2, 3, 4, IJ). Given the task goal initial state Ip. and corresponding abstract state sg = ABSTRACT(20), bilevel planning uses Al planning techniques (e.g., Helmert [1I]) to generate candidate abstract plans. An abstract plan a sequence of ground operators (Wi....,.4) w,,) where 9 S Sn and s; (w; for 1 S S n. The corresponding abstract state sequence (s1....., 8n) serves controller sequence (C,....,C., C..) provides plan sketch, where all that remains is to refine the plan by "filling in" the continuous parameters. We sample continuous parameters O for each controller starting from the first and checking if the controller achieves its corresponding subgoal s;. If we cannot sample such parameters within a constant time budget, we backtrack and resample, and eventually even generate a new abstract plan. We adapt previous work [2] to learn neural network samplers after learning operators (see ($D) the appendix for more details).
sequence of subgoals for low-level planning. The
4 Learning Operators from Demonstrations
To enable efficient bilevel planning, operators must generate abstract plans corresponding plan sketches that are likely to be refinable into low-level controller executions. Given demonstrations D consisting of goal action sequence corresponding state sequence T,, and set of objects O, we wish to find the simplest set of operators such that for every training task, abstract planning with these operators is able to generate the plan sketch corresponding to the demonstration for this task.
Specifically, we minimize the following objective:
J(2) A (1 coverage 2)) + Xcomplexity(1))
where coverage the fraction of demonstration plan sketches that planning with operator set f able to produce and complexits small enough so that complexity is never decreased the expense of coverage.
the number of operators. In our experiments, set be
We will approach this optimization problem using hill-climbing search (Algorithm 2), which ben- efits from having a more fine-gra -reiggained interpretation of coverage defined over transitions within a trajectory instead of entire trajectories. To develop this definition, we will first introduce the notion of necessary atoms.
Definition 4.1. Given an operator set Q, and an abstract plan (wi..... wx) in terms of Ω that achieves goal g, the necessary atoms at step are On A g. and at step 0 < are
a; 1 Pi+1 U (au+1 \ Eit.).

---

In other words, an atom is necessary if it is mentioned either in the goal, or the preconditions of future operator in a plan. If operators in f2 model each of the necessary atoms for every timestep, then these operators must be sufficient for producing the corresponding abstract plan via symbolic planning. Moreover, each necessary atom set minimal in that no atoms can be removed without violating either the goal or future operator's preconditions. Thus, we need only learn operators to model changes in these necessary atoms.
Given necessary atoms, we can now define what it means for some sequence of operators to be consistent with some part of a demonstration.
Definition 4.2, An abstract plan suffix (wk..... w.,) using operators from set S2 and with necessary atoms (0k--1,.... an) is consistent with a demonstration (z, τ) for goal g and timesteps S l S n, where I- = (To...... (n)) and T7 = (ui...., Un)) if for l S I n, (1) the states are consistent: a; S ((ABSTRACT(2--1..4 w)) ABSTRACT(Z;); and (2) the actions are consistent: if the controller for b2 C,, then u, (0) for some 0.
Note that consistency is defined at the level of individual transitions within a demonstration. Thus, it is useful even if s2 does not contain sufficient operators to produce a plan that is consistent with an entire demonstration. Given this notion, we can now define coverage for particular demonstration.
Definition 4.3. The demonstration coverage, 17(2, (T, T, 9, O)) of demonstration (T, τ) of length n for goal y and objects O by operators Ω, is number from 0 to n indicating the length of the longest consistent suffix for the demonstration that can be generated with ground with objects O.
Now, we define fine-grained notion of coverage for optimization via hill-climbing:
coverage D) E1) nww, (z,, U, g. O))
Computing coverage implicitly requires that we compute an abstract plan suffix, as well as nec- essary atoms (in order compute 7)). To do this efficiently, will leverage the fact that know the necessary atoms for the final timestep of every demonstration (they simply the goal atoms!), and compute necessary atoms and plan suffixes for previous timesteps via a preimage backchaining procedure that starts at the goal and works backwards given operators in Ω. Pseudocode for this algorithm is shown in Algorithm 1. We pass backward through each transition in a demonstration trajectory while attempting to choose an operator in Ω to cover (per Definition 4.2) the transition and then updating the transition's necessary atoms using Definition 4.1. If there are multiple operators in Ω that all satisfy the conditions of Definition 4.2, we use heuristic that selects the operator that best matches this particular transition (see ($C) in the appendíx for details).
Hill-Climbing Search. We perform a hill-climbing search on ((2) (Equation 1), starting with the empty operator set. We have two search-step generators: the IMPROVECOVER AGE generator picks a demonstration that is not completely covered by the current operator set and proposes a change that will increase the coverage term of the objective, and the REDUcHCOMPLEXITY generator simply proposes delete operators to decrease the comp exmpleeiiee ty term.
EMpROVECOVERAAE generator. Given a candidate operator set Ω preimage backchaining is used to identify an abstract plan suffix of length & with corresponding necessary atoms (ak-1, on,) that is consistent with a demonstration (z, 7 for goal g, where т = (zo...., 2x) and T = (u,,..., u,). and where < < n. Since < n, we know the transition (I 1 22-1) with necessary atoms ag-1 is not covered by any operator in Ω. To improve the coverage term in our objective, we wish to generate least one new operator to cover this transition without uncovering any others. Definition 4.2 gives the following constraints on the new operator with grounding wt
1. The controller C must match u4-1-
2. P S 8k-2, i.e. the preconditions must be satisfied the previous state.
3. ((sR-2 \ (8-1) nak-)) C Et, i.e., the add effects must include the added necessary atoms.
4. F() C $k-1, i..., the delete effects should remove all atoms in 8k-2 but not in Sk-1-
5. must at least include one variable (with corresponding type) for each of the objects in 1-4.
5

---

Environment
Ours
LOFT
LOF TeFTlypayyy
C1
CI+QE
GNN Shoot
GNN MF
Paining
98.80 (0.42)
(0.00)
(0.91)
99.00 (0.31)
(1.4)
(6.30)3.3))
(0.29)
Satellites
93.40 (3.52)
0.00 (0.00)
34 (5.28)
91.60 (2.68)
95.20 (1.30)
40.40 (3.04)
11.00 (1.44)
Cluttered
100.00 (0.00)
17.20 (5.46)
(0.00)
17.40 (5.52)
92.80 (0.90)
98. (0.63)
98.60 (0.63)
Screws
100.00 (0.00)
0.00 (0.00)
0.00 (0.00)
(0.00)
50.00 (15.81)
95.60 (3:05)
95.80 (3,07)
Cluttered Satellites
95.20 (0.75)
0.00 (0.00)
0,00 (0.00)
1.60 (0.61)
6,00 (1.57)
4.80 (1.27)
0.00 (0.00)
Cluttered Painting
99.20 (0.42)
0.00 (0.00)
0.00 (0.00)
0.00 (0.00)
0.00 (0.00)
4.60 (1.16)
0.00 (0.00)
Opening Presents
100.00 (0.00)
0.00 (0.00)
83.00 (10.77)
83.00 (10.77)
28.00 (5.96)
0.00 (0.00)
Locking Windows
100.00 (0.00)
0.00
90.00 (4.47)
(4.42)
0.00
0.00
Collecting Cans
77.00 (11.75)
0.00 (0.00)
0.00 (0.00)
1.00 (0.94)
0.00 (0.00)
0.00 (0.00)
Sorting Books
69.00 (11.61)
0.00 (0.00)
0.00 (0.00)
(0.00)
0.00
0.00 (0.00)
Table I: Percentage success rate on test tasks for all domains. Note that BEHAVIOR domains use training and lesting sizes of 10 tasks, while all other domains use 50 tasks. The percentage standard error is shown in brackets. All entries within one standard error from the best mean success rate are bolded.
These constraints exactly determine w's controller, but there are many possible choices for the other components that would satisfy 2-5. One straightforward choice would be to optimize prediction eгror over the single transition ($k-2; 1k-1, 5k-1) and necessary atoms ak-1, but this would lead to hyper-specific operator.
Instead, we construct more general operator that covers this transition as well as other transitions in the data set. Specifically, we w's ground controller C uk-1 satisfy condition 1 ahove, and ground add effects E+ ((s \ sk-1) aa-))) to satisfy condition 3. We then construct the controller C' and add effects E+ by 'lifting': i.e, replacing all objects that appear in C' and E by variables of the same type. We record the substitution necessary between the variables and objects for this transition as 6,. The operator's arguments are then the union of the variables that appear here. To find preconditions and delete effects that satisfy conditions 2 and 4, we will use each substitution o, associated with each transition in D to lift each transition т, replacing all objects with arguments while discarding any atom with objects not in 6,. Following Chitnis et al. [2], the preconditions are then: P + -r=(e..:)) ö, (s:). Similarly, the atomic delete effects are: (s:). Finally, the operator's s quantified delete effects are set to be the set of predicates that appear in the current predicted abstract state, but not the observed abstract state: F(si,() ()) \ 8:-1- The final delete effects are the union of the atomic and quantified.
)))) \. FT + (r=(su1s(+1))D (1++1))
Some additional bookkeeping is necessary once this new operator has been induced to guarantee that this generator decreases the coverage term (we must recompute preconditions and delete effects for all current candidate operators, since their respective transition datasets might have changed, etc.), and we detail these, along with a proof of termination, in the appendix ($B.1).
REDUCE COUPLECTTLYY generator. Given candidate operator set Ω we simply delete a single operator from the set and then recompute preconditions and delete effects for all remaining operators as described above (since now, their associated transition datasets D. might change).
5 Experimental Results
Our experiments are designed to empirically answer the following questions:
Q1. Does our approach learn operators capable of generalizing to novel goals and situations with different objects than seen during training?
Q2. How effective is bilevel planning using our learned operators vs. existing baselines?
Q3. How does the performance of our approach vary with the amount of training data?
For additional analyses, including learning efficiency and scalability, complexity of learned opera- tors, task planning efficiency, and ablations of our approach, see (5G) and ($H) in the appendix.
Environments. We provide high-level environment details, with specifics in the appendix (SE). We deliberately include few simple environments to highlight differences between methods.
Painting: a challenging robotics environment used by Silver et al. [3, 1]. A robot in 3D must pick, wash, dry, paint, and then place various objects.
Satellites: a 2D environment inspired by the Satellites domain from Bacchus [12]. but aug- mented with collisions and realistic sensing constraints. See appedix (SF) for further details.

---

Cluttered ID: simple environment where the robot must move and collect objects cluttered
along a 1D) line. An object can only be collected if it is reachable.
Screws: a 2D environment where the agent controls a magnetic crane and must pick specific
screws from clutter to place them into a receptacle.
Cluttered Satellites: same as "Satellites", except readings must be taken from multiple objects.
Cluttered Painting: same Painting, except the robot be next to many objects at time.
BEHAVIOR-I00 Tasks: set of complex, long-horizon household robotic tasks simulated with realistic 3D models of objects and homes [5]. In Opening Presents, the robot must open number of boxes. In Locking Windows, the robot must close a number of open windows. In Collecting Cans, the robot must pick up a number of empty soda cans strewn amongst the house and throw them into trash can, Sorting Bookx, the robot must find books living room and place them each onto cluttered shelf.
Approaches. We describe the approaches we compare against, with details in the appendix (SF).
Cluster and Intersect (CI) [1]: Induces a different operator for every set of unique lifted effects. LOFT [3]: Optimizes prediction error, but with a more general class of preconditions. We include version that only learns from demonstrations (LOFT) and version that collects ad- ditional transitions (2500) in each domain (LOFT+
in the original work. Collecting replay data in BEHAVIOR was intractable due to the size and complexity of the simulation.
Cluster and Intersect with Quantified Delete Effects (CI + QE): A variant of Cluster and Inter- sect that is capable of learning operators that have both atomic and quantified delete effects. It first runs Cluster and Intersect, and then induces quantified delete effects by optimizing predic- tion error via a hill-climbing search.
GNN Shooting: Trains an abstract GNN policy with behavioral cloning and uses it for trajectory optimization. This is inspired by baseline from [1], see (SF) in the appendix for details.
GNN Model-Free: Uses the trained GNN above, but directly executes the policy.
Experimental Setup. For the non-BEHAVIOR environments, we run all methods on up to 50 training demonstrations. For BEHAVIOR environments, we use 10, since collecting training data in these complex environments is very time and memory intensive. Training demonstrations were collected by a hand-coded 'oracle' policy that was specific to each environment. All experiments were conducted on quad-core Intel Xeon Platinum 8260 processor with 192GB RAM limit, and all results are averaged over 10 random seeds. For each seed, we sample set of evaluation tasks from the task distribution T. The evaluation tasks have more objects, different initial states, and more atoms in the goal than seen during training. Our key measure of effective bilevel planning is success rate within a timeout (10 seconds for non- OO-BEEHAIIOR environments, 500 seconds for BEHAVIOR environments, and 1500 seconds for "Sorting Books": the most complex environment).
Learning from Few Demonstrations
Eval Tuaks Salved
100
Painting
Simple
50-
Screws
Satellites
Clutbered Painting
10
20
50
Demonstrations
Figure 3: Data-efficiency of main approach.
Results and Analysis. Table I shows the success rate achieved testing tasks for all our envi- ronments and methods. Our method solves many more held-out tasks within the timeout than the baselines, demonstrating its efficacy even in environments where an assumption on downward refin- ablity might not hold. In our simpler environments (Painting and Satellites; Opening Presents and Locking Windows for BEHAVIOR), the controllers cause a small number of changes in the abstract state, and baseline approaches (CI, CI + QE) perform reasonably well. In all the other environments, the controllers cause large number of changes in the abstract state, and the performance of opera- tor learning baselines degrades substantially, though GNN-basclines perform well on Cluttered ID
7

---

and Screws. Despite the increased complexity, our approach learns operators that are resilient to the lack of downward refinability, enabling bilevel planning to achieve a substantial test-time success rate under timeout. The performance in Collecting Cans and Sorting Books is especially notable; all baselines achieve negligible success rate, while our approach achieves a near 70% rate on testing tasks. Upon investigation, we found that failures are due to local minima during learning for certain random seeds. Figure 3 shows method's testing success rate function of the size of its train- ing set for our non-BEHAVIOR environments. Our approach's ss performance improves with more data, though as the dataset size increases, the impact of additional data on performance reduces.
6 Related Work
Our work continues a long line of research in learning operators for planning [13, 14, 15, 16, 17, 18, 19, 20, 21]; see Arora et al. [22] for a recent survey. This previous work focuses on learning operators from purely discrete plan traces in the context of classical (not bilevel) planning.
Other work has considered learning symbolic planning models in continuous environments [23, 24, 25, 26, 27, 28, 29, 30, 31J, but typically the interface between symbolic planner and low-level poli- cies assumes downward refinability, which requires that every valid high-level plan must be refinable into low-level steps [32], critical assumption we do not make. Therefore, efforts are most di- rectly inspired by LOFT [3] and learning Heuro-Symbolie Relational Transition Models [2], which optimize prediction error to learn operators for bilevel planning. Like our method, LOFT performs a search over operator sets, but commits to modeling all effects seen in the data and searches only over operator preconditions. We point out the limitations of optimizing prediction error in com- plex robotics environments, and take inspiration from Silver et al. [1] who show that optimizing for planning efficiency can enable good predicate invention. We include LOFT and Cluster and Intersect (used by [2. 1]) as baselines representative of these previous methods in our experiments.
The bilevel planner used in our work can be viewed as search-thert saarch-then-samppee sample solver for (TAMP) [33, 34, 10]. This bilevel strategy allows for fast planning in continuous state and action spaces, while avoiding the downward refinability assumption. To that end, our work also contributes to a recent line of work on learning for TAMP. Other efforts this line include sampler learning [35,36, 37, 38), heuristic learning [39, 40, 41], and abstract plan feasibility estimation [42, 43].
Limitations, Conclusions and Future Work
Our method assumes that the provided predicates v comprise good state abstraction given the task distribution for operator learning. With random or meaningless predicates, our approach is likely to learn complex operators such that planning with these unlikely to outperform non- symbolic behavior-cloning baselines. Fortunately, prior work [1] suggests such 'good' predicates can be learned from data. There is no guarantee that overall hill-climbing procedure will con- verge quickly; the IMPROVECOVERAGE successor generator is especially inefficient and has high worst-case computational complexity. In practice, we find its learning time to be comparable or faster than baseline methods in our domains (see ($G) in the appendix), though this may not hold in more complex domains where a very large (c.g. greater than 100) number of operators need to be learned. Additionally, our IMPROVECOVERAGE successor generator is rather complicated, and there is perhaps a simpler and more efficient way to optimize the coverage term in our objective.
Overall, we proposed new objective for operator learning that is specifically tailored to bilevel planning, and search-based method for optimizing this objective. Experiments confirmed that op- erators learned with our new method lead to substantially better generalization and planning than those learned by optimizing prediction error and other reasonable baselines. Important next steps include learning all components necessary to enable bilevel planning, including the predicates [1], controllers [4], and object-oriented state-space, as well handling stochasticity and partial observ- ability. We believe that pursuing these steps will yield important progress toward solving sparse- feedback, long- horizon decision Decision-making problems at scale via modular learning.
8

---

Acknowledgements
We gratefully acknowledge support from NSF grant 2214177; from AFOSR grant FA9550-22-1- 0249; from ONR MURI grant N00014-22-1-27440 12-224000 from ARO grant W911NF-23-1-0044, from the MIT-IBM Watson Lab; from the MIT Quest for Intelligence; and from the Boston Dynamics Ar- tificial Intelligence Institute. Nishanth, Willie, and Tom are supported by NSF Graduate Research Fellowships. Any opinions, findings, and conclusions recommendations expressed in this mate- rial are those of the authors and do not necessarily reflect the views of our sponsors. We thank Jorge Mendez, Aidan Curtis, and anonymous conference reviewers for helpful comments earlier drafts of this work.
References
[1] Silver, R. Chitnis, N. Kumar, W. McClinton, T. Lozano-Pérez. L. P. Kaelbling, and Tenen- baum. Predicate invention for bilevel planning. Proceedings of the AAAI Conference Arti- ficial Intelligence, 2023.
[2] R. Chitmis, T. Silver, J. B. Tenenbaum, T. Eozzano-Pérez, and L. P. Kaelbling. Learning neuro- symbolic relational transition models for bilevel planning. In The IEEF/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022.
[3] Silver, R. Chitnis, J. Tenenbaum, L. Kaelbling, and Lozano-Pérez. Learning sym- bolic operators for task and motion planning. In The JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021.
[4] T. Silver, A. Athalye, J. B. Tenenbaum, T. Lozano-Perez, and L. P. Kaelbling, Learning neuro- symbolic skills for bilevel planning. In Conference on Robot Learning (CoRL), 2022.
[5] Srivastava, C. Li, M. Lingelbach, R. Martin-Martin, F, Xia, K. E. Vainio, Z. Lian, C. Gok- men, S. Buch, Liu, S. Savarese, H. Gweon, J. Wu, and Fei-Fei. BEHAVIOR: Benchmark for everyday household activities in virtual, interactive, and ecological environments. In Con- ference Robot Learning (CoRL), 2021.
[6] J. Brady, R. S. Zimmermann, Y. Sharma, B. Schölkopf, J. von Kügelgen, and W. Brendel. Provably learning object-centric representations. The International Conference on Machine Learning (ICML), 2023.
[7] M. Chang. T. Ullman, A. Torralba, and J. Tenenbaum. compositional object-based approach to learning physical dynamics. In The International Conference Learning Representations (ICLR), 2017.
[8] T. Kurutach, A. Tamar, G. Yang, J. Russell, and P. Abbeel. Learning plannable representa- tions with causal infogan. Advances in Neural Information Processing Systems, 31, 2018.
[9] A. Howe, C. Knoblock, McDermott, A. Ram, M. Veloso, Weld, W. SRI, A. Barrett, D. Christianson, et al. PDDL- the planning domain definition language. Technical Report, Tech. Rep., 1998.
[10] C. R. Garrett, R. Chitnis, R. Holladay, B. Kim, T. Silver, L. P. Kaelbling, and T. Lozano-Péree Integrated task and motion planning. Annual review of control, robotics, and autonomous systems, 2021.
[11] M. Helmert. The fast downward planning system. The Journal of Artificial Intelligence Re- search (JAIR), 2006.
[12] F. Bacchus. Aips 2000 planning competition: The fifth international conference on artificial intelligence planning and scheduling systems. Al magazine, 2001.

---

[13] G. L. Drescher. Made-up up minds: constrrcctivist approach to artificial intelligence. MIT press, 1991.
[14] E. Amir and A. Chang. Learning partially observable deterministic action models. The Journal of Artificial Intelligence Research (JAIR), 2008.
[15] N. Krüger, Geib, Piater, R. Petrick, M. Steedman, F. Wörgötter, Ude, T. Asfour,
D. Kraft, D. Omrčen, et al. Object-action complexes: Grounded abstractions of sensory-motor processes. Robotics and Aulonomous Systems, 2011.
[16] T. Lang, M. Toussaint, and K. Kersting. Exploration relational domains for model-based reinforcement learning. The Journal of Machine Learning Research (JMLR), 2012.
[17] K. Mourao, L. S. Zettlemoyer, R. Petrick, and M. Steedman. Learning strips operators from noisy and incomplete observations. arXiv preprint arXiv:1210.4889, 2012.
[18] H. M. Pasula, L. Zettlemoyer, and L. P. Kaelbling. Learning symbolic models of stochastic domains. The Journal of Artificial Intelligence Research (JAIR), 2007.
[19] C. Rodrigues, Gérard, C. Rouveirol, and H. Soldano. Active learning of relational action models. In International Conference on Inductive Logic Programming. 2011.
[20] S. N. Cresswell, T. L. McCluskey, and M. M. West. Acquiring planning domain models using LOCM. The Knowledge Engineering Review, 2013.
[21] D. Aineto, S. Jiménez, and E. Onaindia. Learning strips action models with classical planning.
In The International Conference Automated Planning and Scheduling (ICAPS), 2018.
[22] A. Arora, H. Fiorino, Pellier, M. Métivier, and S. Pesty. review of learning planning action models. The Knowledge Engineering Review, 2018.
[23] N. Jetchev, T. Lang, and M. Toussaint. Learning grounded relational symbols from continuous data for abstract reasoning. In ICRA Workshop on Autonomous Learning, 2013.
[24] E. Ugur and J. Piater. Bottom-up learning of object categories, action effects and logical rules: From continuous manipulative exploration to symbolic planning. In The IEEE International Conference on Robotics and Automation (ICRA), 2015.
[25] A. Ahmetoglu, M. Y. Seker, Piater, E. Oztop, and E. Ugur. Deepsym: Deep symbol gen- eration and rule learning from unsupervised continuous robot interaction for planning. arXiv preprint arXiv:2012.02533. 2020.
[26] M. Asai and A. S. Fukunaga. Classical planning in deep latent space: Bridging the subsymbolic-symbolic boundary. In The AAAI Conference on Artificial Intelligence (AAAI), 2018.
[27] B. Bonet and H. Geffner. Learning first-order symbolic representations for planning from the structure of the state space. In The European Conference on Artificial Intelligence (ECAI), 2020.
[28] M. Asai and C. Muise. Learning neural-symbolic descriptive planning models via cube-space priors: The voyage home (to STRIPS). arXiv preprint arXiv:2004. /2850, 2020.
[29] E. Umili, E. Antonioni, F. Riccio, R. Capobianco, D. Nardi, and G. De Giacomo. Learning symbolic planning domain through the interaction with continuous environments. ICAPS PRI. Workshop, 2021.
[30] G. Konidaris, P. Kaelbling, and T. Lozano-Pérez. From skills to symbols: Learning symbolic representations for abstract high-level planning. The Journal of Artificial Intelligence Research (JAIR), 2018
10

---

[31] A. Curtis, T. Silver, J. B. Tenenbaum, T. Lozano-Pérez, and L. Kaelbling. Discovering state and action abstractions for generalized task and motion planning. In The AAAI Conference on Artificial Intelligence (AAAI), 2022.
[32] B. Marthi, S. J. Russell, and J. A. Wolfe. Angelic semantics for high-level actions. In The International Conference on Automated Planning and Scheduling (ICAPS), pages 232-239, 2007.
[33] S. Srivastava, Fang. L. Riano, R. Chitnis, S. Russell, and P. Abbeel. Combined task and motion planning through extensible flanner-independenn interface layer. In The IEEE In- ternational Conference on Robotics and Automation (ICRA), 2014.
[34] N. T. Dantam, Z. K. Kingston, S. Chaudhuri, and L. E. Kavraki. Incremental task and motion planning: A constraint-based approach. In Robotics: Science and Systems (R:SS), 2016.
[35] R. Chitnis, D. Mannelld-Menell, A. Gupta, S. Srivastava, E. Groshev, C. Lin, and P. Abbeel. Guided search for task and motion plans using learned heuristics. The IEEE International Conference on Robotics and Automation (ICRA), 2016.
[36] A. Mandalika, S. Choudhury, O. Salzman, and S. Srinivasa. Generalized lazy search for robot motion planning: Interleaving search and edge evaluation via event-based toggles. In The International Conference on Automated Planning and Scheduling (ICAPS), 2019.
[37] Z. Wang, C. R. Garrett, L. P. Kaelbling, and T. Lozano-Pérez. Learning compositional models of robot skills for task and motion planning. The International Journal of Robotics Research (IJRR), 2021.
[38] J. Ortiz-Hare J.-S. Ha, D. Driess, and M. Toussaint. Structured deep generative models for sampling constraint manifolds in sequential manipulation. In Conference on Robot Learning (CoRL), 2022.
[39] W. Shen, F. Trevizan, and S. Thićbaux. Learning domain- iinniinnnddeeeddett planning heuristics with hypergraph networks. In The International Conference Automated Planning and Scheduling (ICAPS), 2020.
[40] B. Kim and L. Shimanuki. Learning value functions with relational state representations for guiding task-an uas-and-motion planning. Conference on Robot Learning (CoRL). 2019.
[41] C. Paxton, Raman, G. D. Hager, and M. Kobilarov. Combining neural networks and tree search for task and motion planning challenging environments. In The IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS), 2017.
[42] D. Driess, J.S.. Ha, and M. Toussaint. Deep visual reasoning: Learning to predict action sequences for task and motion planning from an initial scene image. In Robotics: Science and Systems (R:5S), 2020.
[43] M. Noseworthy, C. Moses, I. Brand, S. Castro, L. Kaelbling, T. Lozano-Pérez, and N. Roy. Active learning of abstract plan feasibility. In Robotics: Science and Systems (R:SS), 2021.
[44] M. Helmert and Domshlak. Landmarks, critical paths and abstractions: what's the difference anyway? In The International Conference on Automated Planning and Scheduling (ICAPS), 2009.
[45] W. Battaglia, J. B. Hamrick, V. Bapst, A. Sanchez-Gonzalez, Zambaldi, M. Malinowski. A. Tacchetti, D. Raposo, A. Santoro, Faulkner, et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint 110.:v8800.016..... 2018.

---

A Bilevel Planning Algorithm Details
Algorithms and 4 provide pseudocode and additional details necessary for high-level and low-level search respectively in bilevel planning (Algorithm 5).
GENABSTRACTPLAN/S, 9. Ω Pabaraa)
< GROUND([))
// Search over ground operators from s0 to goal (returns top plans)
f f- SEARCH(SO, 9, Q, Rabstract)
return ă
Algorithm 3: This is GENABSTRACTPLAN which finds high-level plan by creating operators for all possible groundings then uses search to find Rabmmat plans. It returns plans ã, which the REFINE procedure helow will attempt to turn into executable trajectories.
REFINE((R, Tp, V, E, phamlles)) state & (ZSTRACT(2))
// While current state is not goal, sample and run current operator current state and check ground atoms. If is passes continue if not backtrack.
curriax +- 0
while currid, < ltn(ft) do samples[currial] +
samples[currip
statccurent Q +- [[currax]
S2..A Q.
π (curraa - Q.C
curTiax - currias +1
if I. h...Ciiiiaalllslaleeereree
then
statenext Simulate Q.C)
stateexpected; +- मे [curraa if state S Estate then can_cconinue_ong + True if curria (sneseeeton)) then
10
11
return sucess,
else
canContinueOn False else
13
canContinucOn + False not can CantinnueeOn then curriax & currid -
14
if samples[currias = max maxplesples then return failure,
16
return success, π
Algorithm 4: This is REFINE which turns task plan á into sequence ground skills. It gets the state and operators from à and adds the controller with newly sampled continuous parameters to и.. After this it checks to see the added controller is initiable from the current state in the plan and we simulate the skill execution verify it reached the expected state predicted next in ft If the controller is not initiable fails the expected atoms check we backtrack and resample a new continuous parameter for this controller until either reach the number of samples way. the performance of the low-level controllers in the environment strongly impacts the overall planning process.
successfully refine our final controller. In this
12

---

BILEVELPLANNINGCC Zo. 9. V, 22 Σ)
// Paraneters: rasc Rsamples
80 < ABSTRACT(20)
// Outer Planning Loop
for á in GENABSPLAN(Su, 9. S2, Rabstract) do
// Inner Refinement Loop
if REFINE(, To. Ω Pssamples) succeads w/ return
Algorithm 5: Pseudocode for bilevel planning algorithm, adapted from Silver et al. [1]. The inputs are objects O initial state æo, goal g. predicates V, operators f2, and samplers 52; the output a plan The outer loop GENABSPLAN generates high-level plans that guide our inner loop, which samples continuous parameters from our samplers concretize each abstract plan then the found plan ı is returned as the solution; if fails, then the outer CAANABSTRACPLAS continues.
plan л. the inner loop succeeds,
IMPROVECOVERACFEC. D)
COVinit: Da. Tunc: Clunc f- COMPUTECOVERAGEE(0, D) if covini then
return Q
COV.urr f- COVinit
Q + Ω
while covcarr 2 cOVini do
Whew f INDUCHOPTOCOVER Tune Orunc) O REMOVEPRECANDDEL.E
Wnew
(D w1 .......... (6,, .....,,..,,,,, 6,,) < PARTITIONDATA(22, D)
N + INDUCEPRECANDDELLEES((.
(D ... Dw.)
(5
10
2 + S' U ENSURENECATTOMSS AT (Wnows Da)
(D W1 ... Dwi). (6T, ................................ 6,,) + PARTITIONDATA(I), S + INDUCPPRICANNDELLEPSSSS
12
(Dwi... ...))
(o,,..., 8r,))
13
COVeur, Day Tuncy Oune COMPUTECOVERAGE(TY D)
14
Q' + "RUNNNNIDDATTTPPAATTSSSSSSS
return !!'
Algorithm 6: Pseudocode for improve successor generator. The inputs are set of operators Q of all training demonstrations D, the corresponding set of training tasks Teii The output set of operators ?' such that coverage(R") < coverage()))
B Detailed Description of Successor Generators
B.1 IMPROVECOVERAGE
The pseudocode for the improve coverage successor generator shown in Algorithm Given the current candidate operator set Ω training demonstrations D and corresponding tasks Tuains first attempt to compute the current coverage of Ω on D. We do this by calling the COMPUTE- COVERAGE method, This method simply calls Algorithm on every demonstration (z, T in D (the set of objects and goal required by Algorithm are obtained from the training tasks). The CoMputECoVERACC. method then returns the number of covered transitions (cov dataset of necessary atoms sequences for each demonstration Algorithm 1 is able to cover (Da). the first uncovered transition encountered (Tunc = (sk, Ug+1, 8k+1)), and the corresponding necessary atoms for the transition (Oune). If the number of covered transitions is the same as the size of the training dataset, then all transitions must be covered and the coverage term in objective (Equation
"The total number of transitions in abstract plan suffixes that Algorithm is able to find when run on each demonstration in D.
13

---

1) must be O. We thus just return the current operator set f2 with no modifications. Otherwise, we compute a new set of operators Q with a lower coverage value.
To generate S2' we first create new operator with preconditions, add effects and arguments set to cover the transition Tune and corresponding necessary atoms O'inc. The operator's ground controller C(O) u is determined directly from the transition's action Uk+1- The operator's ground add effects are set to be E+ = (#k+1 \ (k) O Ouune. The controller and add effects are lifted by creating a variable U2 for every distinct object that appears in CU E". The operator's 's arguments T are set to these variables.
Next, we must induce the preconditions and delete effects of this new operator Whew To this end, add Whew to current candidate set, and partition all data in our training set D into operator specific datasets Dw for each operator in our current candidate set. Since operator preconditions and delete effects depend on the partitioning, we first remove these from all operators that are not Whew
We perform this partitioning by running the FINDBESTCON SISTENTOP method from Algorithm 8 on this new operator set for every transition in the dataset, though we do not check the condition pred S 81+1, since the operators do not yet have delete effeets specified. While performing this step, we save mapping 67 from the operator's arguments to the specific objects used to ground it for every transition in the dataset (this will be used for lifting the preconditions and delete effects of cach operator below). We assign each transition to the dataset as- sociated with the operator returned by datasets (Dw, ..........!
We return the operator specific 8,,).
Dw.). ),
as well as the saved object mappings for each transition
We now induce preconditions and delete effects using (Dw, ... Du,) and (ōr)...., 6z,). Before we do this, we delete any operator whose corresponding dataset is empty. Similar to Chitnis et al. [2], we set the preconditions to: P N (s:). We also set the atomic delete effects to ET + -r=terree-iEED (5141) \ (s:). For every transition (ss; U4+1, $1+1), let sprrd (s, \ UE.. Then, set Smispred Ur=(vreeuilED $11 \\ $1+1. We induce quantified delete effect for every predicate corresponding to atoms in Smispred. We then set each operator's delete effects to be the union of E and the quantified delete effects.
Now that all operators have preconditions and delete effects specified, we must ensure that the newly-added operator (Wnew) able to satisfy the necessary atoms for each of its transitions in D Wan Recall that we set the operator's add effects to be the necessary atoms that changed in the first uncovered transition Tune Given the way partitioning is done (specifically the conditions in the FINDBESCCONSISTENTT
method in Algorithm 8), we know that these add effects must satisfy 04+1 C $141 U E+ for all transitions (st, U4+1, 81+1) E Duze with corresponding necessary atoms a;+1 for state 8-41- However, the delete effects may cause the necessary atoms to become violated for certain transitions: i.e, 04++1 E (#++1's JuE.. For every such transition, we let miss = a0+11 ((s++1 \ ) U EE). We then create new operator witliss by copying all components of Whew. and adding lifted atoms from miss to both the preconditions and add effects. We modify the operator's arguments to contain new variables accordingly. This now ensures that the necessary atoms not violated for any transition in Dumar We add these new operators to the current candidate operator set.
After having added new operators to our candidate set in the above step, we must re-partition data and consequently re-induce preconditions and delete effects match this new partitioning (lines 11-12 of Algorithm 6). We now have new operator set that is guaranteed to cover the transition Tune that was initially uncovered. We check whether this new set achieves a lower value for the coverage term of our objective, and iterate the above steps until it does.
Finally, after the while loop terminates, we remove all operators from D that have associated datasets that are empty. This corresponds exactly to removing operators that are not used in any abstract plan suffix computed by COMPUTECOVERAGEE and are thus unnecessary for planning.
Proof of termination To see that the main loop of Algorithm 6 is guaranteed to terminate, consider that the operator set Ω strictly grows larger every loop iteration (no operators are deleted). Since
14

---

the predicates are fixed, there is finite number of possible operators. Thus, at some finite iteration, O' will contain every possible operator. At this point, it must contain an operator that covers every transition and the loop must terminate.
Anytime Removal of Operators with Null Data In the IMPROVECOVERAGE procedure as illus- trated in Algorithm 6, we only prune out operators that do not have any data associated with them after the main while loop has terminated. However, we note here that we can remove such operators from the current operator set (S2') any time during the algorithm's loop.
This property arises because the amount of data associated with particular operator will only decrease over time. To see this, note that (1) the number of operators in Y' only increases over time, and (2) data assigned to the 'best covering' operator as judged by our heuristic in Equation 3. Given a particular operator w at some iteration of the loop, suppose there are d transitions from D associated with it (i.e, |Pwl = d). During future (i.e loop iterations, new operators will be added to Ω. For any of the d transitions in Dus these new operators either be worse match (in which case, the transition will remain in Dw), become associated with the new operator). Thus, for any operator w, once there is no longer any data associated with it, there will never be any data associated with it, and will simply be pruned after the while loop terminates.
better match (in which case, the transition will
As result, we can prune operators from our current whenever there is data associated with them. We do this in implementation, since it improves our algorithm's wall-clock runtime.
B.2 REDICECOMPLEXITY
REDUCECOMPLEXITYC.. D, Timin) N' + DELETHOPERATOR(()
(Dw, ... Dun .........., 6r,) + PARTITIONDATA({/ D)
S + IADUCEPPECANDEELLPSSSSS (Dw, ... Dun.),
(ōn....,,65))
return! !!'
Algorithm 7: Pseudocode for our reduce-compl exity successor generator. The inputs set of operators Q the set of all training demonstrations D, the corresponding set of trainign tasks Texn The output set of operators ?' such that complexity(2") complexity(f).
The pseudocode for our reduce-cooe complex-o generator is shown in Algorithm 7. As can be seen, the generator is rather simple: we simply delete an operator from the current set (DeLEtEOPERATON) and return the remaining operators. Since we've ve changed the operator set, must recompute the partitioning and re-induce preconditions and delete effects accordingly.
This generator clearly reduces the complexity term from our objective (Equation 1), since |S2 < |S]]
C Associating Transitions with Operators
FINDBESTCONNISTENNTE((.... 8,, 8141, a, O))
Roon +- () for w E do
2
for w GETALLGGRUUDDNGSS O) do
144 + ((s, E-)
if C AND C pedd
AND #11 8 AND 30 39:C0C C(() u, then
pred
Que f- Pewn Uw 1 * 1 then
return FINDBESTCOVER(O. Si41))
Algorithm 8: Pseudocode for the FindBestConnssseentO) helper method used in Algorithm
15

---

A key component of our algorithm is the FINDBESTCOVER method from Algorithm 8, which in turn is used in Algorithm 1 and the PARTITIONDATA method of Algorithm The purpose this method is to associate a transition with a particular operator when multiple operators satisfy the conditions necessary to 'cover' it. Intuitively, we wish to assign transition to the operator whose prediction best matches the observed effects the transition. We can do this by simply measuring the discrepancy between the operator's add and delete effects, and the observed add and delete effects in the transition. We make two minor changes this simple measure that are appropriate to our setting. First, we only use the atomic delete effects as part of our measure. We exclude the quantified delete effects because these exist in order to enable our operators to decline to predict particular changes in state. Second, we favor operators that correctly predict which atoms will not change. Recall that the ENSURENECATOMSS, AT method in Algorithm induces such operators by placing the same atoms the add effects and preconditions.
Given some transition (si, 14+1, 5:+1). and some ground operator y with atomic delete effects E, our heuristic for data partitioning is represented by the score function shown in equation 3.
K Et
C = E+ K
score [6 \ (8i+1 \ *x]]+
(3)
[(s+1 \ st) ||+
(E (s: \ (1+1))[++
(st \ (1+1)) E. C
Once all eligible operators have been scored, we simply pick the scversgscoring operator to associate with this transition. multiple operators achieve the same score, we break ties arbitrarily.
D Learning Samplers
In addition to operators, we must also learn samplers to propose continuous parameters for con- trollers during plan refinement. We directly adapt existing approaches [2, 1] to accomplish this and leamn one sampler per operator of the following form: a (zz 01..... 0₂) = So [x[o1] e ...... O æ(ox)), where zr(o] denotes the feature vector for o in x, the @ denotes concatenation, and 8, the model to treat the problem one of supervised learning on each of the datasets associated with each operator: D. Recall that for every transition (I;, U44 1441,22411 in D.s mapping 5 : + O, from the operator's 's arguments T to objects ground the operator with. Recall also that every action is a hybrid controller with discrete parameters and continuous parameters 0. To create a datapoint that can be used for supervised learning for the associated sampler, we can reuse this substitution to create an input vector 216, (v1) + ... @ x(6(vx)), ), where (Vi...., 2k) = T. The corresponding output for supervised learning is the continuous parameter vector @ in the action U++1-
be learned. Specifically,
Following previous work by Silver et al. [1] and Chitnis et al. [2]. we learn two neural networks to parameterize each sampler. The first neural network takes in æ[o1] @ ... @ x [oo)] and regresses to the mean and covariance matrix of a Gaussian distribution over 9. We assume that the desired distribution has nonzero measure, but the covariances can arbitrarily small in practice. To improve the representational capacity of this network, we learn second neural network that takes in x x(or] () @ x[ox] and 0, and returns or false. This classifier is then used rejection sample from the first network. To create negative examples, use all transitions such that the controller used in the transition matches the current controller, but the transition is not in the operator's 's datasct D...
16

---

E
Additional Experiment Details
Here we provide detailed deseriptions of each of experiment environments. See ($5) for high-level deseriptions and the accompanying code for implementations.
E.1
Screws Environment Details
Types:
-he screw type has features y, he
The receptacle type has features x, Y.
The gripper type has features x,
Predicates:
Pickable (?x0:grippez, ?xl: ree:rtccptalle)
AboveReceptacie :x0::grrppppr ?xl: receptacle)
HoldingScrew (?x0: :rripper, ?xl:screw), ScwewIneeecpttccleeeeee (2x:: ?x1: : receptacle)
Actions:
MoveToScrew (?x0: gripper, ?xl: screw): moves the gripper to be
Near the screw 2x1.
MoveToReceptacll (7x0: gr gripper, 7x1: receptacle): moves the gripper to be (bovecccplace((xdgrripp re.eeccppaccl)
MagnetizeGriipee (2x0: gripper): Magnetizes the gripper at the current location, which causes all screws that the gripper is Near to be held by the gripper. Demagneti i wwweetiieerrppppep.. (?x0: gripper): Demagnetizes the gripper at the current location, which causes all screws that are being held by the gripper to fall.
The agent must make ScrewInReceptacho (?x0: screw, ?xl: recepecet true for a particular screw that varies per task.
Goal:
E.2
Cluttered ID Environment Details
Types:
-The robot type has features x.
- -The dot type has features x, grasped.
Predicates: Next robotr ?x1 :xtt)ot), (2x robot), Grasped (2x0:robot, :dot).
Actions:
MoveGrasp (xx0: robot, ?x1: dot, [move_or.grasp, x]): A single controller that performs both moving and grasping. If If 0.5, then the controller moves the robot to a continuous position y. Else, the controller grasps the dot ?x1 is within range.
The agent must make Grasped (?x0 robot, ?x1 :dot) true for particular set of dots that varies per task.
Goal:
E.3
Satellites Environment Details
Types:
The satellite type has features x, y, theta, instrument, calibration.ch obj obj.id. is calibrated rea id, shoots chem sheots.
17

---

The ob ject type has features id, x, bas_chem x, has chem.y, y.
Predicates: Sees ( ?x1:obfect),
:xl:objec,,
IScalibrateed((20:ssseeel111
HasCamera (?x0:satellite))
Hasshfffareed(200:saattelllll
HasGeiger
(MocttsChemm((xx0sssstt1111 #ascheemX((xX):sateellll)
ShootsChemi (2x0statelttt))))t)t).
HasChemY
?xl:object),
ixtraceeheedaagggkkeee220aataeed ?xl:object), Ge
?xl:objectt.
Actions:
MoveTo (2x0 :satellite, 2x1 2xl:object, [x, y) Moves the satellite ?x0 to be at x, y.
- Ca Calibrate (?x0:satellits) the satellite ?x0 against object ?xl. This will only succeed (i.e, make (2x ?x0.
7x1: :object:: Tries to calibrate
true) if ?x1 the calibration target of
ShootChemx :aat:mmmllla:aalaaaaa1 ssatelitt,, ?x1: object): Tries to shoot pellet of chemical X from satellite ?x0. This will only succeed if ?x0 both has chemical X and capable shooting it.
SheotChemY (7x0 satellite, ?x1 :object):: Tries to shoot pellet of chemical from satellite ?x0. This will only succeed if 7x0 both has chemical Y and is capable of shooting it.
UseInst 2x00sateelite ?x1 objectt: Tries to use the instru- ment possessed by 7x0 on object ?x1 (note that we assume 7x0 only possesses a single instrument).
Goal: The agent must take particular readings (i.e some combination of CameraReadingTaaker (2x0:satelli:: ?x1:object), InfrareddResdiingTaeenn (2x0:satellite, ?x1:objects:
GeigerReadingTenn satt:satlllte,, ?xl:object)) from a specific set of objects that varies per task.
E.4
Painting Environment Details
Types:
The object type has features y, z, dirtiness, wetness, color, grasp, he ld.
The box type has features x, y, color.
The iid type has features open.
The shelf type has features y, color.
The robot type has features y, fingers.
Predicates:
InBox (?x0:obj),
InShelf (?x0:obj)
IsBoxColor :2x0: obi, ?xl :box), ?x1 ?x1:shelf), GripperOpen (?x0 : robot), NotOnTable (x00: :bb)). HoldingTop : Holding (bx0:obj), IsDirty (2x0:obi). IsClean (bx0:obj).
IsShelfColor (?x0:obj,
OnTable (2x0:obj),
HoldingSide (2x0:obj),
TsWet (?x0:obj),
IsDry (?x0:obji
Actions:
18

---

Pick (?x0 robot, ?x1 :ob), [grasp]] ): picks up particular object, if grasp > 0.5 performs a top grasp otherwise side grasp.
Wash (?x0 :obbtt): washes the object hand, which is needed clean the object. Dry rox0:robot): drys the object in hand, which is needed after you wash the object.
Paint (?x0: : robot, [colox]) ) paints the object in hand a particular color specified by the continuous parameter.
Place (2x0 robot, [x, y, 2)): places the object in hand at particular x, 7. location specified by the continuous parameters.
OpenLid (?x0: robot, ?xl: opens a specific lid, which is need to place objects inside the box.
Goal: A robot in 3D must pick, wash, dry, paint, and then place various objects in order to get InBox (?x0:ob) and IsBoxColor (?x0:obj. ?x1 :box), or InShelf (2x0:00 :bbj) and IsShelfColos (?x0:obj, shelffelf) true for par- ticular goal objects.
E.5
Cluttered Painting Environment Details
Types:
The object type has features x, y, z, dirtiness, wetness, color, grasp, held.
The box type has features x, y, crlor.
The lid type has features open.
The shelf type has features y, color.
The robot type has features x, y, fingers.
Predicates:
InBox (?x0:
InShelf (2x0
IsBoxColor (2x0:obj,
?x1 :box),
?xl:shelf), GripperOpen robbtt
OnTable (2x0:obj),
(ot00 (od.inngooopp Holdinggiie
Holding
IsWet (?x0:obj). IsDizty(?x0:0bj). IsClean (2x0: along with RepeatedNexfTo Predicates: NextToBox (?x0: robot, roxo:
IsDry (?x0:obj).
NextTo (?x0:zooot ?xl:obj),
?xox)
(2x:::0shh@((0000000:::0: ?xl:shelf), NextToTable (2x0: ?x1:table).
Actions:
Pick (?x0 : robot, ?x1: obbj [grasp]
picks up particular object, grasp > 0.5 it performs a top grasp otherwise side grasp.
Wash (?x0 robot): washes the object hand, which is needed clean the object.
Dry (?x0 robot): drys the object in hand, which is needed after you wash the object.
Paint (xx0 : robot, [color]) : paints the object in hand a particular color specified by the continuous parameter.
Place (xx0 robot, [x, y, 21) places the object in hand at particular x, y, 7. location specified by the continuous parameters.
Opennid (?x0: robot, ?x1:lid) opens a specific lid, which is need to place objects inside the box.
MoveToob robotr ?xl:obi, [x] moves particular object with certain displacement x.
19

---

[x] moves to a particular box with MoveToBox r?x0:robot, ?xl:box, certain displacement x.
MoveToShelf (?x0: robott ?xl: Pxl:shelf, [x] ) moves particular shelf with certain displacement
Goal: A robot in 3D must pick, wash, dry, paint, and then place various objects in order to get InBox (?x0:obj) and IsBoxColor (?x0:obj, ?x1 :box), or InShelf (?x0:obj) and IsShelfColos (?x0 :obj. ?xl shelf) true for par- ticular goal objects. In contrast to the previous painting environment, we also need to navigate to the right objects (i.e. all objects are not always reachable from any states). This version of the environment requires operators with quantified delete effects.
E.6 BEHAVIOR Environment Details
Types:
Many object types that range from relevant types like hardbacks and notebooks to many irrelevant types like toys and jars, All object types have features from location and orientation complete list of object types and features see [5].
graspable and open. For
Predicates:
Inside (?x0 (?x0:obj, ?x1: :xl:)bj), OnTop (7x0:obj, ?x1 sobj), Reachable-Nothhe (), HandEmpty () Holding (?x0:obj), Reachable (), Openable (?x0:obj) :ob)), Not-Openable (bx0:ob)), Open (2x0:ob]),
Closed (?x0:obj).
Actions:
NavigateTo (?x0 :obj) : navigates to make a particular object reachable.
Grasp (x0 : obj, [x, y, z]): picks up particular object with the hand start- location specified by the continuous parameters.
ing at particular relative
PlaceOnTop (?x0: obj): places the object in hand ontop another object as long as the agent holding an object and is in range of the object be placed onto.
P Place Ins ide :ox0:bbi) j): places the object in hand inside another object as long as the agent is holding an object and is in range of the object to be placed into.
Open (?x0 tobj) opens specific object (windows, doors, boxes, etc.) if it is 'openable'.
Close (2x0 obj)) ): closes specific object (windows, doors, boxes, etc.) if it is currently in an 'open' state.
Goal: In Opening Presents, the robot must Open (?x0 package) number of boxes of type package around the room. In Locking Windows, the robot must navigate around the house to Close (xx0 window) a number of windows. In Collecting Cans, the robot must pick up number empty soda cans of type pop strewn amongst the house and throw them into a trash can of type bucket. This will satisfy the goal of getting Inside ( :oop ?x1 : bucket) for every soda can around the house. Sorting Books, the robot must find books of type hardback and notebook in living room and place them each onto a cluttered shelf (i.e. satisfy the goal of OnTop (2x0 hardback, 7x1 : s f) and OnTop (?x0: notebook, 7x1: sxl:shelf f) for number of books).
F Additional Approach Details
Here we provide detailed descriptions of each approach evaluated in experiments. For the ap- proaches that learn operators, A search with the Imeut heuristic [44] the high-level planner for bilevel planning in non non-BEHAVIOR environments, and use Fast Downward [11] in
20

---

configuration with minor differences from lama-first as the high-level planner in BEHAVIOR environments, since A search was unable to find abstract plans given the large state and action spaces of these tasks. All approaches also iteratively resample until the simulator f verifies that the transition has been achieved, except for GNN Model-Free, which completely model-free. See ($5) for high-level descriptions and the accompanying code for implementations.
F.1
Ours
Operator Learning: We learn operators via the hill-climbing search described in Section 4.3. For our objective (Equation 1), we the term to be 1//D), where (D) represents the number of transitions in the training demonstrations.
Sampler Learning: As described in Section D. each sampler consists of two neural net- works: generator and discriminator. The generator outputs the mean and diagonal covariance of Gaussian, using an exponential linear unit (ELU) to assure PSD covari- ance. The generator fully-connected neural network with two hidden layers of size 32, trained with Adam for 50,000 epochs with learning rate of 1e-3 using Gaussian nega- tive log likelihood loss. The discriminator is binary classifier of samples output by the generator. Negative examples for the discriminator are collected from other skill datasets. The classifier is fully-connected neural network with two hidden layers of size 32, trained with Adam for 10,000 epochs with a learning rate of le-3 using binary cross entropy loss. During planning, the generator is rejection sampled using the discriminator for up 100 tries, after which the last sample is returned.
Planning:
The number of abstract plans for high-level planning was set to N Nabttratt = 8 for our non-BEHAVIOR domains, and Nansmett = for our BEHAVIOR domains. The samples per step for refinement was set to Namples = 10 for all environments.
F22
Cluster and Intersect:
This is the operator learning approach used by Silver al. [1].
Operator Learning: This approach learns STRIPS operators by attempting to induce different operator for every set of unique lifted effects (See Silver al. [1] for more infor- mation).
Sampler Learning and Planning: Same as Ours (See ($F.1) for more details).
F.3 LOFT:
This is the operator learning approach used by Silver et [3]. We include version ("LOFT+Replay') that allowed to mine additional negative data from the environment match the implementation of the original authors. We also include version ("LOFT") that is restricted to learning purely from the demonstration data.
Operator Learning:
This approach learns operators similar to the Cluster and Intersect baseline, except that it uses search to see it can modify the operators after performing Cluster and Intersect (See Silver et al. [3] for more information).
Sampler Learning and Planning: Same Ours (See ($F.1) for more details).
F.4
CI QE:
A baseline variant of Cluster and Intersect that is capable of learning operators that have quantified delete effects in addition to atomic delete effects.
21

---

Operator Learning: This approach first runs Cluster and Intersect, then attempts to in- duce quantified delete effects by performing a hill-climbing search over possible choices of quantified delete effects using prediction error as the metric to be optimized.
Sampler Learning and Planning: Same as Ours (See 1) for more details).
E5
GNN Shooting:
This approach trains a graph neural network (GNN) [45] policy. This GNN takes in the current state z, abstract state s ABSTRACT(z; VG), and goal g. It outputs an action via a one-hot vector over C corresponding to which controller to execute, one-hot vectors over all objects at each discrete argument position, and vector of continuous arguments. We train the GNN using behavior cloning on the dataset D. At evaluation time, we sample trajectories by treating the GNN's s output continuous arguments as the mean of a Gaussian with fixed variance. We use the known transition model f to check if the goal is achieved, and repeat until the planning timeout is reached.
Planning: Repeat until the goal is reached: query the model on the current state, abstract state, and goal to get a ground skill. Invoke the ground skill's sampler up to 100 times to find subgoal that leads the abstract successor state predicted by the skill's operator. successful, simulate the state forward: otherwise, terminate with failure.
Learning: This approach essentially learns a TAMP planner in the form of GNN. Follow- ing the baselines presented in prior work [2], the GNN is standard encode- encode-process- proces--dcoodee architecture with 3 message passing steps. Node and edge modules are fully-connected neural networks with two hidden layers of size 16. We follow the method of Chitnis et al. [2] for encoding object-centric states, abstract states, and goals into graph inputs. To get graph outputs, we use node features to identify the object arguments for the skill and global node with a one-hot vector to identify the skill identity. The models are trained with Adam for 1000 epochs with learning rate of 1 and batch size 128 using MSE loss.
F.6 GNN Model-Free:
A baseline that uses the same trained GNN above, but evaluation time, directly executes the policy instead of checking execution using This has the advantage of being more efficient to evaluate than GNN Shooting, but is less effective.
G Additional Experimental Results and Analyses
Environment
Ours
LOFT
LopT-replay
CI+ OF+
GNN
Painting
(3.58)
9126611.41)
135.73 (6.15)
70.95 (5.07)
(5.86)
(290.91111199)
Satellites
19.38 (7.83)
52.73 (18.35)
438.44 (51.6
23.29 (5.38)
15.96 (4.70)
1625 (218.88)
Clutter ID
17.98 (1.06)
(17.68)
366.89(146.000 (146.09)
62.68 (14.89)
28.58 (3.68)
1164.92 (84.74)
Screws
1.31 (0.04)
143.60 (49.10)
5712.80 (7 (736.84)
0.32 (0.02)
708.98 (1023.02)
1369.59 (68.44)
Cluttered Satelltes
16.12 (0.55)
353.67 (52.78)
902.99 (148.22) 22)
107.04 (11.94)
87.24 (10.49)
3043.62 (285.27)
Cluttered Painting
131.68 (5.05)
1699 52 (216.71)
7364.03 (532.67)
470.32 (40.38)
2788. 74 (1330
4615.70 (334,11)
Opening Presents
(27.72)
(23.66)
(17.01)
locking Windows
16.77 (1.55)
62.55 (10.12)
61:71 (8.95)
45.51 (5.74)
319.09 (7.61)
Collecting Cans
3728.73 (9544.75)
1520.93 (354,20)
576.89 (100.57)
781.38 (350.46)
2121 86 (120.51)
Sorting Books
1981.799144400.0
6423.03 (602.444)
1528.18 (111.18)
$359.99 (170.46)
Table 2: Learning times seconds on training data domains. Note that BEHAVIOR domains (bottom
4) use training set sizes of 10 tasks, while all other domains use training and testing set sizes of 50 tasks. The standard deviation is shown in parentheses.
We have already established that our approach learns operators that lead to more effective bilevel planning than baselines. In this section, we are interested in comparing our approach with basclines on three additional metrics: (1) the efficiency of high-level planning using learned operators, (2) the efficiency of the learning algorithm itself, and (3) the simplicity operator sets we learn. We also ablations of our method to investigate the importance of optimizing the complexity term, well downward-refinall to our method.
22

---

Hades Coudee Scrmus with Drews
Mores Created Selidilte tes Demes
Vedes Oraace Chalored Painiing with terwo
UPFT
Node Dosted
14'
Crotte
30
30P
37
3>*
101
19
13
13
121
Figure 4: Nodes Created by Operator Learning Approaches. We show scatter plots of the nodes created (x-axis) for each operator learning approach (y-axis). We also include violin graph to visualize the density of points throughout the graph. If bilevel planning failed, we set the nodes created to 10 for non-BEHAVIOR domains and 10" for BEHAVIOR domains. Our approach achieves a low number of nodes created across when compared to baselines in most domains.
Environment
Ours
LOFT
LOFT+replay
CI
CI+ QE
Painting
10.00 (0.00)
13.60 (0.80)
19.20 (0.39)
11.00 (0.00)
10.20 (0.40)
Satellites
7.40 (0.79)
10.90 (1.44)
33.80 (3.70)
10.40 (1.20)
9.30 (0.9)
Cluiter ID
2.00 (0.00)
7.
16.10 (2.11)
7.10 (1.64)
3.00 (0.44)
Screws
4.0 (0.00)
(1.988
91.14 (5.11)
14.80 (1.98)
4.80 (0.97)
Cluttered Satellites
7.00 (0.00)
19.80 (2.60)
59.60 (4.45)
16.30 (1.10)
13.9 (0.83)
Cluttered Painting
13.00 (0.00)
28.00 (0.00)
157.8 (6.49)
25.20 (2.31)
20.70 (1.61)
Opening Presents
2.30 (0.90)
10.80 (2.99)
10.80 (2.99)
9.80 (1.83)
Locking Windows
2.00 (0.00)
6. (0.70)
6.10 (0.70)
4.70 (0.64)
Collecting Cans
6.10 (5.37)
57.40 (9.43)
52.90 (8.41)
13.40 (2.33)
Sorting Books
14.70 (7.57)
76.70 (5.62)
75.80 (5.79)
Table 3: Average number of operators learned for all domains. Note that BEHAVIOR domains (bottom 4) use training set sizes of 10 tasks, while all other domains use training and testing set sizes of 50 tasks. The standard deviation is shown in parentheses.
Figure 4 shows the nodes created during high-level planning for each of our various environments and operator learning methods. We can see that operators learned by our approach generally lead to comparable or fewer node creations during planning when compared to baselines. In many of the environments where baseline methods are able to achieve number of points with fewer node cre- ations Cluttered ID, Opening Presents, and Locking Windows our method has a significantly higher success rate.
Table 2 shows the learning times for all methods in all domains Our approach achieves the lowest learning time in 7/10 domains. Upon inspection of our method's performance on the 'Locking Windows' and 'Collecting Cans" domains, we discovered that the high average learning times because of few outlier seeds encountering local minima learning, yielding large and complex operator sets (this is the reason for the extremely high standard deviation).
NNot that there is no entry for 'CI+ QE' for sorting books because learning exceeded the memory limit of hardware (192 GB)
23

---

Environment
Ours
No complexity
Down Eval
Painting
98.80 (1.33)
98.80 (1.33)
26.60 (6.52)
Satellites
93.40 (11.14)
81.20 (19.40)
84.20 (16.88)
Cluttered ID
100,00 (0,00)
100.0 (0.00)
100.00 (0.00)
Screws
100.00 (0.00)
100.00 (0.00)
100.00 (0.00)
Cluttered Satellites
95.20 (2.40)
94.80 (2.72)
94.00 (3.22)
Cluttered Painting
99.20 (1.33)
99.00 (1.84)
23.80 (7.56)
Opening Presents
100,00 (0,00)
100.00 (0.00)
100.00 (0.00)
Locking Windows
100.00 (0.00)
100.00 (0.00)
100.00 (0.00)
Collecting Cans
77.00 (37.16)
75.00 (38.30)
75.00 (38.80)
Sorting Books
69.00 (36.73)
52.00 (34.00)
67.00 (37.2)
Table 4: Percentage success rate for our original method, well as ablations where we set the À parameter from Equation to on tasks (No complexity), and enforce downward refinability evaluation time (Down Eval) for all domains. Note that BEHAVIOR domains use training and testing set sizes of tasks, while all other domains use training and testing set sizes of 50 tasks. The percentage standard deviation is shown in brackets.
Table shows the number of operators learned for all operator learning methods in all domains. Our approach learns the lowest number of operator sets across all environments and massively out performs other approaches this metric in Collecting Cans, and Sorting Books. These results further highlight our ability to learn operator sets that efficient for high-level planning, but also simpler and therefore, more likely to generalize to new environments.
Table 4 shows the success rate of our method when the À parameter from Equation set to (the 'No complexity" column), thereby effectively removing any impact on the optimization from the complexi ty term in the objective. In most environments (Painting, Cluttered ID, Screws, Clut- tered Satellites, Cluttered Painting, Opening Presents, Locking Windows, and Collecting Cans), this change has minimal impact on the method's success rate. However, in two enviromments (Satellites and Sorting Books), the change causes a significant reduction in success rate. Upon inspection, we found that our approach learned a number of very complex operators in these domains. When À was set to be greater than 0, our approach would delete a number of these operators, but in this ablation, was unable and thus planning performance suffered. We can conclude from these experiments that optimizing the complexity term is key component of approach particular domains. In fact, we believe that more aggressively optimizing the complexity term, perhaps by increasing the sophistication of our REDUCECOMPLENCE nificantly even on the two complex BEHAVIOR tasks, since we found that our approach learned overly-complex operators for few random seeds.
step, could enable us to improve performance sig-
Table 4 also shows the success rate of our method when bilevel planning only allowed to produce 1 abstract plan during refinement. This effectively enforces that the learned operators must yield downward-refinabll plans. This causes significant reduction in success rate in many environments, showing that the ability to evaluate multiple abstract plans is important to plamning with operators learned by our method. Indeed, all the environments that exhibit significant reductions in success rate do not to our knowledge admit a downward-refinal high-level theory over the provided skills and predicates.
H Learned Operator Examples
Finally, we provide operator examples to demonstrate our approaches ability to overcome overfiting to specific situations. Figure 5 shows a comparison of the operators learned with Open in Opening Packages environment and Navigate in Collecting Cans environment across our approach and 'CI QE' (the most competitive baseline in these environments). As shown, by optimizing prediction error 'CI QE' learns number of operators describe the same amount of transitions that is cov- ered by the single operator approach learns. Upon inspection, 'CI QE learns overly specific operators when trying to cluster effects that try to predict the entire state to the point where "Quan-

---

packapec
Open-packagee:
Argunents: [?x0=psckege]
Argunents: [2x0 [=xc:package]
Preconditions:
Preconditione:
closed- package (7xepggggee)))) handempty().
closed-packagel (ax:ppccaa)))
handempty().
spenable-package 7x8 Px@:package).
(aanable-package(2400 (aaekkae(
reachable-pacee packkleeeppegeeeeeeeeee (7x0: packkgee)
reachable-package [>xB:package]]
Add Effects: [open-package( :packege)]
Effects: lopen-package packpee]
Delete Effects: (elosed-packagel PxB: packkge))
Delete Effects: plakkeep (aackage(xx0:)))) [xe:packagee]
Quantified Delete Effects:
Quantified Delete Effects: []
certooppaccege-rrore [foor]
Controller: Open-package( (arokagee(xx::ooo) (xappaage)
Controller: Open- pack-ee spackage),
Open-package1:
Arguments: foxe:room_ ?x1 :package]
Preconditions:
handempty(), floooomoflorrroo roon
not-openable-rone
ontop- stto--package-roon_ffloe floorffooorr_o :package, 7x8:romm,floor)
Effects: (toolll-rrroo_ooooor2eeeeeee floor)]
Effects: packaae((x11
cacoop-paaage-aeree roono roon_floor (Px1:packaaa 1x8:room, floor)
Quantified Delete Effects: []
Controller: Open-package (xalppacka))))))
NavigateTo-pop0:
NavigateTo-popé:
Argunents: [?x8:pop]
Argunents: [7x0:bed, ?x1:pop]
Preconditians:
Preconditions:
handempty().
handempty().
Effects: [reachable-pop( (7x8:pop)
hot-openable-pogg(h111a (pop))
Delete Effects:
ontop-pop-bed(?<11 (po1:ppp. ?x8:bed),
Quantified Delete Effects: [
reachable-bedi [xachable-bedd(BBbbbeb))
ontop-pop-ppp
Add Effects: [reachable-pop [pxp]
reachable-bede
Delete Effects: [reachable-bed( :bdd]
sacchable-bucker
Quantified Delete Effacts:
reachable-pan
Controller: NavigateTo-popl popp1(1pppp)
Controller: NavigateToo-opp>>x::p00
NavigateTo-pop1:
Argunents: [?x8:pop]
Preconditions:
handempty(),
(ht--pponabll--ooo
reschabbe--pop(XXeppe
Add Effects: []
Dalete Effects: []
Quantified
Controller: NavigateTo-popp
NavigateTo-pop2:
Argunents: [7x0:pop]
Preconditions:
handempty().
Add Effects: [reachable-poa(hkk :pop)]
Delete Effects:
Quantified Delete Effects: []
Controller: Navigatelo-poot pppp(?x::pp))
Figure 5: Operator Comparison. Operators learned after our approach (left) and 'CI+QE' (right), for Open in Opening Packages environment (top) and NavigateTo in Collecting Cans Cans environment. Our approach learns fewer operators that generally simpler, and thus conducive effective bilevel planning and generalization.
tified Delete Effects' are not fully utilized. For the full set of operators learned by our algorithm on the 'Sorting Books' task, see Figure 6.
25

---

trag-astteboic:::
Argumeta:: [bs6.ottsbosk)
rotesssk/7d
Add Effecte: bnl.-rrg-etttaooi [4P-ooreteek]]
Ovantified belets Effecte:
coffee tuble, not.bock-ooo Floor)
Costroller: Grasp resebookk netabook)
ieipsteloomtteeode
Praconditisee: (nardeepty() met-sgennsi--reeeeee potpeesnns tod rnotebstt]
Oelete effectsl []
balets Effecte: (rachttle-ssrrd..m gane mastable-cotfee, taste, renthable-tarhhst, nachitle-atrboll reachstle-shetr meachableetree.ggr
Contrallar: (to
Argunente: (Se:caale ]
ren-pprrallletttttaaaaeeee meachalll-stttt Add Effects: [handappty)), enter hardaack ahulf(1xt dhardbb, 1x8:nne2f]] quesifial Effeate:
dardback(hh (harsbock))
castroller: Flaosd 117 shelf)
Phco0kttp-ahh211:
Presendi [heldarg nntetoon motetook)) vet-spprable-set seperetlle-ettttoteee setebool inttelook); rot-operable (Hansssse))))) reseteble wtw.lle-aaelttreeeeeee (1)
Effects: (andaaptyy)), ortaz-resabbskkateet insttaabe,,, 1a8:ane2r1)
belate Offects: (nsidding-sttebnn (ottbbo)))))))
Oventilied fied belete tifeets:
Argunents: (Sachardbasc)
(handumptr()
burdaash shardacc)), reschuble bardbcck(sM:hhhrraekkk)
Ad Effects: (holdarg hardteck (idthurdackk
[hanowntty) reachable
harchwik)]
quectified telets (aaaactthe-aarrbee narcback)
coddas--oooffee centi-tantarkk-ooe woackk-ooooooo.......... [flor]
Controller: Gruup tar--arrdddeeh
Neviptteoosshe//:
[tee:shef]]
Preconditione: inst-operablee-hvii htw.-chhii/oooooetttttt (naschabee-thhff (shtt)))
Quentified fied belete tifeets: (wrtop hurducck soffee. Luble, reschable bourtlyues, reschable www.w/.aalll..... resclablle harduci, reachable notslonk,
Costrollar:
hurdbeckd:
Argunents: Totmaardbaai)
(hanteepty(). est-opeeat bebandrack) hardbark)]
Add Effects: (neachable (hrddee-(hi:itrddeeee
gusesified belete effeats: [rwwatte-boorr.pe.. mewrable-sssfeeeennee. reaclable-rardbank nestadle--strbokk rewdaste--shelr reatrable-stee.gee
Novigstel-ttrrtte (Naktaa
Figure 6: Sorting Books learned operators.
26


Reviews:
[{"review_id": "bAA7UL5i1Lp", "reviewer": "Reviewer_sXyD", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper is on bi-level planning in continuous-state/action spaces. Here, a high-level abstraction is used plan, which then guides low-level decisions. Existing bi-level planning methods often struggle in tasks where many objects in the environment change as a result of actions. This is because such planners must learn operator descriptions which track every single change between abstract states, and thus become unwieldy.\n\nThe insight of this paper is that operators need only describe the set of partial changes necessary to accomplish specific planning goals. To this end, they develop a formal procedure for distinguishing which changes are actually \"necessary\", and learn partial operators from a small batch of demonstrations via a hill-climbing search algorithm.\n\nOn experiments over the Behaviour-100 benchmark, along with a variety of additional tasks, the paper shows that their algorithm's success rate significantly exceeds many existing methods, including LOFT, CI, and GNNs.", "summary_of_recommendation": "This paper makes a solid technical contribution to bi-level planning, backed up by solid experiments. I am happy to recommend for acceptance. In the camera-ready, I would like to see some small tweaks to narrative clarity to make some of the mathematics easier to follow for the casual reader.", "technical_quality": "Excellent: Very convincing and thorough.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "Mentioned in main review: Minor questions around the relationship of downward refinability to the concrete steps of their algorithm/definitions; a few questions around experimental analysis and the relationship between demonstration quality / task-type to success.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper represents a solid technical contribution to bi-level planning. Its central insight (that you can get away with operator descriptions that only track necessary changes rather than the full set of abstract state changes) is novel, and has a solid grounding both in its mathematical description and experimental results.\n\nThe experiments themselves are thorough, with a wide range of benchmarks and sensible recent baselines. One thing that bothered me a little: It seems that the \"generalization\" of the learned operators depends pretty heavily on *which* demonstrations it is trained on, and what the goals of those demonstrations were. I would have liked to see a little discussion or experiments on this (Though to be fair, they do mention a dependency on the *number* of demonstrations...just not which).\n\nAdditionally, for two of the experiments (Collecting Cans and Sorting books), their algorithm gets around a 30% failure rate. The analysis concludes that this is just \"some random seeds for these tasks get stuck in local minima\", but I am wondering if there is more to say about this? What is it about these tasks that make them more susceptible to getting stuck in local minima? Is this something that their algorithm might be able to overcome in some way?\n\nI also noted that, very briefly in the related literature and a sentence in the experimental results, the authors mention the concept of \"downward refinability\" as a critical assumption used by many bi-level planners. They mention that their algorithm does not depend on this assumption and thus has a big advantage. I was surprised therefore to not see any mention of this concept in the main methods section of the paper, or how it arises in their algorithm. \n\nMy main small issue with this work is clarity: Particularly in sections 3 & 4, I found it a struggle to keep hold of an intuitive understanding of how each component fit together. Some of this is down to space limits, as much of the mechanical details of algorithmic operation is sequestered to a large set of appendices. However, I think that it would improve the main text significantly to either have some concrete examples, or English-language \"in other words...\" parts to make the mathematical definitions (e.g., 4.1 - 4.3) and discussions of complexity & coverage less difficult to follow.\n\n", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning."}, "extra_text": {}}, {"review_id": "UTPmdVdE91", "reviewer": "Reviewer_uCVm", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposed a method that learns symbolic operators (a list of symbolic pre-conditions and effects for high-level actions). The set of possible predicates that can appear in the operators are assumed to be known. The operators are extracted from given demonstrations that have explicit goals.\n\nThe main contributions are:\n- a backchaining procedure that identifies relevant predicates in the demonstrations (different steps of the demonstrations include a large number of predicates that are irrelevant to the task and that need to be filtered out),\n- a hill-climbing search that identifies the best set of operators to achieve planning best, and\n- a rigorous verification of the method in many complex simulated environments compared with strong baselines.\n\nThe main argument of the paper is learning the operators based on their planning performance rather than their prediction accuracy is necessary and bring high gain in planning.", "summary_of_recommendation": "\nIn short, the paper studies an important and challenging problem, proposes a novel technical solution, rigorously verifies the method in a wide range of tasks and environments, and shows that the proposed method significantly outperforms a number of baselines, including [1], which is SOTA and the closest competitor. The theoretical contribution can be highlighted better.\n\nUpdate: The novelty of planning metric is more clear now. This work is not incremental but uses an idea that was already explored before. If the planning-driven learning idea were first used in this paper, I would have recommended strong acceptance. Now, I will keep my weak accept recommendation.", "technical_quality": "Excellent: Very convincing and thorough.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "- Please clearly address and discuss the planning-metric based approach followed in [1] and rephrase the main contributions of this paper. Please also include a discussion in the experimental evaluation section as well.\n- Please provide an ablation where the method only relies on the backchaining procedure (the current method without hill-climbing).\n- Please provide justification for the generalization argument.\n- Please provide an ablation where the method only relies on the backchaining procedure (the current method without hill-climbing).\n- Please provide justification for the generalization arguments.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "- The paper is well-written, with clear motivation, a method that well-addresses the motivation, extensive experiments, and experimental results that show the proposed method has significant quantitative and qualitative advantages.\n- Learning symbolic operators from experience in a self-supervised way is a very challenging problem. It is also crucial as it enables efficient AI planners in robots operating in a continuous sensorimotor world. Learning from demonstrations (rather than unorganized action executions) makes it easier to learn, but still, it is a challenging task that is rarely studied by researchers. \n- The authors argue that the core contribution of the method is to \"learn operators based on planning performance rather than prediction accuracy.\" They cite [1] and even use [1] as one of the baselines. However, [1] also discusses that operators should be learned based on their planning performance rather than prediction performance. [1] also uses a planning-based metric in learning operators. The authors did not mention this. Although the performance of the current method is shown to be significantly better than [1], this aspect makes the theoretical contribution of the current paper look weaker.\n- The authors mention that F encodes a non-deterministic abstract transition model. However, the learned operators are deterministic.\n- The authors argue that their method \"strongly generalizes over initial states, actions, and goals\" and \"generalizes to held-out\" tasks. I could not find a justification for this claim in the main manuscript or the Appendix.", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}, {"review_id": "Rtk4MZ18Swg", "reviewer": "Reviewer_i5Gu", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This work proposes a novel approach for learning symbolic operators to enhance bilevel planning in complex robotics tasks. The objective is to maximize the performance of the planning algorithm by focusing on necessary changes within high-level states. The key innovation, IMPROVECOVERAGE generator, efficiently identifies transitions not covered by current operators and generates new ones to improve overall coverage.  It leverages preimage backchaining to identify abstract plan suffixes within demonstrations that are not fully covered by the current set of operators, which aims to generate new operators and modify existing ones to cover specific transitions. The authors propose a novel method for constructing more general operators that cover multiple transitions in the dataset. By leveraging preimage backchaining and careful substitution, the operators are induced with appropriate preconditions and delete effects. ", "summary_of_recommendation": "Based on the technical appendix and the main content of the work, it is evident that the authors have presented a comprehensive and well-structured study on bilevel planning algorithms. The technical appendix provides detailed descriptions of the algorithms, operators, samplers, and environments used in the experiments, which adds clarity and facilitates replication. The authors have also compared their proposed approach with several baselines, offering a thorough evaluation of its effectiveness. However, with limited knowledge in the field, some detailed aspects of the lower-level controller remain unclear. With the necessary revisions, this work has the potential to make a valuable contribution to the field of bilevel planning.\n", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Fair: Major revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "Due to the reviwer's limited knowledge in the field, there are mainly experiment and implementation evaluation questions remain to be clarified:\n\nQ1: Impact of Lambda Term: The authors mention the choice of the λ term in the objective function (Equation 1); however, they do not thoroughly discuss its impact on overall performance and sensitivity to different values. It would be helpful to include ablation studies concerning the λ parameter in Eq.1 to understand its effect better.\n\nQ2: Complexity of IMPROVECOVERAGE: The computational expense of running the IMPROVECOVERAGE successor generator is not clearly addressed, and this could potentially lead to scalability issues in more complex domains. It would be beneficial to compare the computational cost of this work with related approaches to assess its efficiency.\n\nQ3: Size of the Demonstration Dataset: Could the authors provide information on the scale of the demonstration dataset? Specifically, the number of trajectories or operators within the dataset. Additionally, it would be insightful to investigate how the performance of the bilevel planner is affected by varying amounts of demonstrations.\n\nQ4: Details of Lower Level Controllers: The specifics of the lower-level controllers and their technical construction are not entirely clear. Further elaboration on this aspect is needed to better understand the transition model. Particularly, when performing planning at the abstract level, it would be crucial to know if the success of lower-level controllers is taken into consideration, as this could impact the overall planning process.\n", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Quality:\nThe work is well constructed. The authors present a well-structured and comprehensive study that addresses the challenging problem of learning symbolic operators for bilevel planning in complex robotics tasks. The research is backed by a sound theoretical foundation and is supported by empirical evaluations on various environments. \n\nClarity:\nThe main idea is well and explicitly introduced. However, for readers without extensive field knowledge, some parts may require frequent cross-referencing of the cited works. It would greatly enhance the paper's accessibility if the authors provided a more detailed background introduction, offering readers a better understanding of the context. Additionally, the scattered mathematical formulas throughout the paper can be challenging to follow, and it would greatly improve readability to consolidate all the symbols' meanings in a dedicated section.\n\nOriginality:\nThe IMPROVECOVERAGE generator is a novel algorithm that efficiently identifies abstract plan suffixes not covered by current operators and generates new ones to improve coverage. Additionally, the careful consideration of necessary changes within high-level states sets this work apart from conventional approaches.\n\nSignificance:\nThe significance of this work lies in its practical implications for solving long-horizon robotics problems with continuous state and action spaces. By learning symbolic operators that are better suited for high-level search and generalization, the approach enables efficient and effective bilevel planning in complex environments. The research contributes to advancing the field of task and motion planning, which is critical for real-world robotic applications.\n\nStrengths:\n\nNovel Objective: The focus on maximizing planning performance rather than prediction accuracy is a significant strength, as it aligns with the ultimate goal of efficient and effective planning in real-world scenarios.\n\nIMPROVECOVERAGE Generator: The introduction of this algorithm is a major strength, as it efficiently identifies necessary changes and generates more general operators, leading to improved coverage and planning success.\n\nEmpirical Evaluations: The extensive experiments on various complex environments demonstrate the effectiveness and generalization capabilities of the proposed approach, adding credibility to the findings.\n\nWeaknesses:\n\nData Requirements: The method relies on a set of predicates and demonstrations for operator learning. While it is effective with sufficient data, the approach's performance might be limited with sparse or insufficient data.\n\nComputational Complexity: The IMPROVECOVERAGE generator, although effective, may suffer from computational inefficiency when the operator set becomes large or in more complex environments. Further optimization might be necessary.", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning."}, "extra_text": {}}, {"review_id": "H8pa19tEKe", "reviewer": "Reviewer_wErB", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper focuses on learning symbolic operators in a manner that could be advantageous for low-level planning. Previous approaches learn operators that predict all the changes in state after the application of action.  The authors claim that learning to predict the changes that are relevant for high-level task planning will improve the overall planning performance. An objective function and a learning method are introduced for learning such a partial transition model. The validity of these claims is supported by conducting appropriate experiments.", "summary_of_recommendation": "The motivation and technical approach is well presented. The paper introduces a new way of approaching the problem of operator learning. In my opinion, theoretical contributions hold more significance than real robotic experiments for papers of this nature. Thus, the strengths of the paper surpass its weaknesses.", "technical_quality": "Excellent: Very convincing and thorough.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "1) How scalable is this approach w.r.t to the complexity of the domain? Did you try some experiments, or do you have any plot that gives an idea about the learning time scales w.r.t the number of operators that needs to be learnt? \n", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Novelty:** As far as I know, the proposed objective for learning operators in terms of consistency with the given demonstration is novel. \n\n**Clarity:** The aim and the overall idea are clearly explained, especially the introduction is good. However, the notations are a bit complex to understand. \n\n**strengths**\n1) The paper is very well written and has high-quality technical content. It is presented in a clear and easy-to-understand manner.\n2) Learning operators from low-level demonstrations in a way that produces a feasible plan sketch for guiding low-level planning is indeed an interesting idea.\n3) The video provided in the supplementary provides a very good overview of the approach.  \n4) The experiments are well-designed to address the specific questions under consideration.\n\n**weakness**\n1) No real robot experiments\n2) The hill climbing search that is used to learn the operators may not scale for complex operators and environments.\n3) The paper spends most time motivating and explaining the objective. However, a very compact explanation is provided on how to optimize the objective in the actual search performed. \n4) At certain points, the introduction presents a general idea, yet the subsequent text lacks specific guidance on its implementation. For example,  \n    In section 4, the paper states, \"To improve the coverage term in our objective, we wish to generate at least one new operator ω, and *potentially modify existing operators to cover this transition without uncovering any others*\".  However, the text below only describes the process of learning a new operator to improve coverage, without providing an explanation of how or when existing operators can be modified to achieve this objective.\n    In the introduction, the paper states, \"we focus on the problem of learning operator descriptions from very few demonstrations given a set of predicates, an accurate low-level transition model, and *a set of parameterized controllers (such as Pick(x, y, z)) that serve as primitive actions.*\" However, how to learn samplers for controllers is not included in the main text. Even though previous approaches have been adopted for this problem, stating it somewhere explicitly in the main text will help. \n\n", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes an improvement to bilevel planning with symbolic operators and neural samplers. The paper proposes a novel approach to learning operators that only model changes necessary for abstract planning to achieve specified goals. The paper is well written. In simulation experiments, the proposed approach outperforms baselines.
Decision: Accept (Poster)