Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper exploits pre-defined symbols to discover skills that enable symbolic and low-level plan generation. In an environment with continuous states and actions, assuming pre-defined symbols, symbol mappings, predicates, object ids, transition functions, and unsegmented demonstrations, the proposed system discovers skills that encode symbolic operators (precondition and effect predicates) and skill policies.  Importantly, the skills that encode <precondition, effect> symbols can be achieved via several low-level goals, which brings flexibility in planning: The system can produce multiple symbolic plans in the outer loop and checks out with which detailed policy the symbolic plan is possible in the inner loop. The method is sound, verified in a number of tasks, and compared against baselines. The problem is an important problem, assuming pre-defined symbols and building up everything on top of the pre-defined symbols bring limitations to the scalability of the method.
strengths_and_weaknesses: It is very challenging, important (and timely) to learn skills that allow robots to make both high-level (symbolic) and low-level reasoning from the continuous sensorimotor experience of the robot. High-level reasoning requires symbols and operators to be learned, and low-level reasoning and execution require learning of detailed flexible policies that are linked with the operator definitions. The authors well-address the latter topic, and in this sense, their contribution is important and significant. However, they need to clarify their contribution with respect to [11] and [12] as detailed below.

Saying this, the differences between the current work and the previous work [11] and [12] are not properly provided. The main contribution is stated as learning policies compared to [11] & [12] where the parametrized policies were pre-defined in the previous work. Stating this in only one line and then describing the Neuro-Symbolic Skills (Section 3) and Bilevel Planning (Section 4) without much referring to it is misleading about the contribution. In [12], Chitnis et al. also partitions skill data, unifies transitions based on symbolic effects, find the preconditions and effects of operators, and learn action sampler. This roughly corresponds to 5.1 and operator learning and sampler learning parts of 5.2 in the current paper. I might be missing the details, however, the only difference of the current paper seems to be in step 5.2 - policy learning; whereas all other sub-parts were realized in some form in [11] and [12].

Therefore, the authors need to explicitly and carefully emphasize the exact contribution at the algorithmic level and at the realization/implementation level. They also need to provide why [11] or [12] were not included in the experiments as baselines. Maybe they might not be directly applicable for different reasons, then these reasons should be clearly pointed out. Additionally, was it not possible to apply [11] and [12] in this problem setting with small modifications? This also needs to be explained.



robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This paper proposed, implemented and verified a sound method for a challenging and important problem. However, it looks incremental in its current form.
issues: - Please clarify the detailed contribution with respect to [11] and [12] in algorithmic and implementation levels.
-  The writing of the paper is clear. However, I am not sure if it requires such extensive and a bit complicated notation. Is it possible to simplify to improve the accessibility to a wider audience?
- The description of KD2 can be improved. It is not clear whether creating multiple symbolic plans or the flexibility in the inner loop is more important.
- Please clarify whether object types were predefined or automatically generated from the beginning and also in 5.1 Lifting subsection.
- In Policy Learning, please provide more details (possibly in the Appendix). Is only u_i learned/predicted? How about u_{i+1:k}, i.e. the parameters of the action execution? Why/how a parameter from a single (arbitrary) time-step represents the policy? Please clarify.
- Is it possible to learn a skill with pre-conditions and effects that include a variable number of objects? As far as I see it is not possible. Is it an important limitation? How to extend the method to address this?
- In the experiments, what happens if the training tasks include 3-4 buttons/cups and test tasks include 2-3 buttons? As far as I see, all 3 object types are required to be included in the preconditions (as they are all common in all training data), and therefore the method will fail with 1-2 objects. Please clarify whether this is the case, discuss whether this is an assumption, and re-discuss the generalization capability of the method taking into account this.
- Achterhold et al. (2022) also learn symbolic operators along and continuous control, but with an RL agent. They also do hierarchical planning. This work can be included in related work or discussion

Achterhold, Jan, Markus Krimmel, and Joerg Stueckler. "Learning Temporally Extended Skills in Continuous Domains as Symbolic Actions for Planning." arXiv preprint arXiv:2207.05018 (2022).
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The proposed method tackles the TAMP abstraction problem in learning from demonstration by starting from a manually defined set of symbolic predicates and learning a parameterized policy in combination with symbolic operators and low level subgoal (operator parameter) samplers. There are simulated results.

"We learn skills that achieve certain symbolic effects (e.g., Grasped(robot, stick))
from states where certain symbolic preconditions hold (e.g., HandEmpty(robot))."

Human programmers provide
- a definition of what a low level state is
- a definition of what low level actions are
- the low level dynamics function
- demonstrations in terms of low level states and actions
- an ontology: values, features, and objects have types [This may be what breaks in the real world, but works in games.]
- predicates which retrieve the values of features. The values of all possible predicates are known. In some cases the predicates seem carefully chosen to make the task work.
strengths_and_weaknesses: This paper is on arxiv, the identity of the authors is public information, and this review was done knowing who the authors are. https://arxiv.org/abs/2206.10680

STRENGTHS

I agree with the general philosophy of this paper
that reasoning, planning, and learning at the symbolic level as
well as continuous levels is useful. 

Good introduction.

KD2: I agree considering alternative "strategies" or abstraction sequences
is an important research area.

CONCERNS (some philosophical)

I disagree with the philosophy
that abstractions should be correct representations of the underlying problem
and should be taken seriously. I prefer approaches where abstract plans are often not executable but
suggest where to search in lower level planning spaces.
In current robot dynamic legged locomotion, hierarchical control is used
and more abstract representations are typically simplified (kinematic, dynamic, actuator, ...) 
models that approximate
the lower levels but the output of the simplified models cannot be exactly obeyed by the lower
levels. The simplified models "suggest" where the lower level planners should search.

I would argue that the failure of early symbolic AI approaches to
robotics had to do with 
needing the abstract plan to be correctly executable.
The proposed sampling-based approach only
samples low-level plans that conform to the abstract plan. Low level
planning approaches that explore a region of the low level planning space
suggested but not limited by a high level plan may be more useful.
Similarly, using imperfect or loose analogies may be a more effective way
of making use of abstractions than treating an abstraction as a "correct"
representation of the original problem. Abstractions where abstract plans work in the abstract
but don't always work or are not executable in the real world, and vice versa, may be the norm.
The role of abstraction may be to merely suggest good
starting points or search directions for low level search.

Another issue with this paper is the manual feature selection.
As Patrick Winston often said, if you have the right features, the
problem is easy, and practically any method will work. If you don't
have the right features, then no method really works well.
I am concerned that this work is doomed to succeed, with small numbers
of carefully chosen relevant features. For example, CupFilled is created
and MADE A CONTACT EVENT to get the approach to work on pouring.
This trivializes that task.
Our experience with good old fashioned AI (GOFAI) suggests that
it has been difficult to find the implementable predicates (symbols) for
real world robotics in the past. Has anything changed?

Could the paper discuss when useful abstractions are likely to exist and be findable, before a problem is solved? Note that after a problem is solved it is often easier to find useful abstractions to solve it.
Is it true that a successful abstract plan (as in line 132) always exists
when a successful low level policy exists? This is not clear to me.
A related question is whether a search in abstract space is guaranteed
to find a successful low level policy that does exist.
What properties of abstractions must hold for this to be true?
It may be that the only abstraction that works is no abstraction at all,
S==X. Consider a parity function, for example.

I am concerned the evaluation tasks as implemented in the simulator
are toy problems.
Gripping objects and sticks, door/door-handle manipulation, using the stick as a tool, and pouring are all trivialized. Small numbers of relevant predicates are available for each task. Do sticks bend or break? Do stick tips slip? Does liquid splash during pouring? These are not
simple skills that we can provide low level policies for that work well in general. As mentioned before,
CupFilled is made a contact event to make pouring work. A simple change to the work is to make all predicates from all task available to each task.
Lots of irrelevant predicates should cause segmentation and partitioning to fail.

The same argument goes for object types and objects. A relatively small number of relevant object types and objects are defined for each task. Are furniture objects? Lights and light switches? Interior decorations like pictures and rugs? How about the floor, wall, and ceiling? How about dust bunnies?
How are the objects involved in a task known? Are they all the objects that were present in the corresponding demonstration? Or just the ones changed (which ignores useful objects like tables)?

I note that demonstrations are generated with handcrafted
environment-specific policies. If doing the demonstrations is a solved
problem, why are we bothering to learn to do these tasks?

All the tests/experiments are done in simulation. What issues
exist concerning the sim-to-real gap when applying these plans and skills on a
real robot?
Will this approach work in the real world? Will it
run out of gas, as previous symbolic approaches to robotics have.
Is this work doomed to succeed because of the tasks and environments chosen to work with and the use of "clean" simulations? Simulated and "symbolic" (often man-made) environments and systems
(such as the ones used as tasks in this work) easily lend themselves to symbolic representations. Messy real world and "natural" systems don't.

I note that object types seem to be pre-defined rather than learned.
It is known what a button is, and that it can be pressed. This is a second manual intervention, in addition to defining predicates, whose implications are not discussed in the paper.
The robot
cannot construct a new object and discover it is a button.
Analogies must be exact, rather than loose or suggestive.
This makes lifting easy and well defined. How is the case handled where an
object has properties like a button, but is not completely like
a button in all ways, or fit the type definition?

"If sampling fails for an operator sequence, the outer loop continues to another sequence, addressing KD2."
Much more information needs to be provided about when the system gives up on
trying to attain a particular subgoal. KD2 should be modified to say "an agent
should be able to EFFICIENTLY consider multiple skill sequences ...".
Considering alternative "strategies" or abstraction sequences is not all that useful if it is not efficient.
The planning algorithm proposed in section 4 is likely to be very inefficient.

A concern is the notion that there are discrete objects. The notion of object
as presented in the paper breaks down with liquids, granular materials,
and deformable materials like peanut butter and clay (plastic materials?).
It also does not handle materials that are consumed, created, or transformed during use
(food, fuel, cooking, ...) well. It is difficult to reason about object construction and
structural failure (fracture, breakage, wear, fatigue, ...).
What does it mean for an object to be wet? Glued or nailed to another
object?

This paper reminds me of early work in machine learning on
symbolic concept learning, such as https://axon.cs.byu.edu/~martinez/classes/778/Papers/Mitchell.ch2.pdf
That work was useful, but eventually superseded with statistical or
numerical methods. Will the same happen to this work?

I agree that segmenting trajectories at contact state changes makes sense.
However, it probably makes sense to have a hierarchy of segments.
At the lowest level one has segments A, B, C ...
The next level might have AB, BC, ...
The next level might have ABC, ...
This helps deal with the case that multiple contact changes lead to a
known state (ball rolls down an inclined plane into a V shaped corner or
onto a floor).
It always ends up at the bottom of the V or on the floor,
but the contact sequences may vary.

The learned subgoal sampler as designed is unlikely to explore out-of-distribution parameters. For example let's say it needs to rotate a bolt or screw in a screw, and in the evaluation/real-world scenario, it happens that it needs to exert way more force than it is used to, simply because there might be more friction or corrosion, how likely is it to apply out-of-ordinary/distribution force? It cannot reason about the possible reasons for the screw being stuck.

How is the "true" goal of a demonstration known? How are irrelevant actions ignored? For example, human breathing is usually ignored in demonstrations, but this is not obvious without strong priors.
How are changes in the environment not caused by the demonstrator ignored (water evaporates or leaks,
objects may move on their own or due to gravity, ...)?
Lots of irrelevant features (predicates) should break this ability to tell what is relevant in a demonstration.

Add and delete operators were designed for features that had true or false values, and does not seem like the most useful semantics for robots.
I don't see how add and delete operators handle cases where a value is incremented in the paragraph on operator learning starting on line 184.
For a feature x with a value 3.2, to add 1.9 to x one doesn't delete x and then add x with the new value.
We don't want x in E- and in E+.
It makes more sense to have a set M of features whose values are modified.
What is the semantics of <e+,e-> = <s'\s,s\s'> when values of features are modified?

SUGGESTIONS

It is much easier to read this paper AFTER reading the supplementary material.
It would be nice if that were not the case. Many comments below (written as I
read the paper first) ask for definitions and examples, which are provided in the 
supplementary material. The paper does not adequately stand alone.
The examples of symbolic
plans and how the underlying primitives work in the supplement are really helpful. It would be great if one of them could be in the paper itself.

I don't understand why the term "neural" is used in this paper.
There is no connection to biological neurons. The neural sub-goal conditioned
policy can be represented by any parameterized function framework,
as line 205 points out.
A subgoal sampler is just something that proposes instances from
a density function, which again can be represented in many ways, either
with a parametric function approximator or a nonparametric representation
which retains all the instances in the training set and picks one or
locally interpolates to propose a candidate.
Is the word neural used because that is the current fad in function
approximation? Is there any evidence that the neural network architectures
used in this paper are superior to other function approximators for the
purposes they are used in this paper? There are only a relatively small
number of training examples (less than 10,000), so the argument for
deep neural networks does not really apply.
I would call what is proposed a hybrid-symbolic-and-numeric algorithm, or a hierarchical algorithm.

Sections 3, 4, and 5 would be easier to read if a simple example was used
throughout. Maybe navigation through a series of rooms, where each room
was an abstract state, and skills existed to find doors, open them, and
move to another room.

It might be useful to note that KD1 implies some lookahead so later (soft and hard) constraints are
taken into account. This is what trajectory optimization and backpropagation
in time do.

I can't figure out what the tasks are just from Figure 1.

An environment (vs. a task) has object types (button), but not specific objects
(button1, button2, ...)?
Or did you want to include the object space O in the environment tuple
< Alpha, O, U, f, Phi >?
The set of objects in a task should be a subset of the set of objects
in the environment (irrelevant objects should be able to be present).

In top-k planning, how are the candidate plans made non-trivially
different? Such a planner could generate plans that have trivial
differences.

The notation in line 165 may not be familiar to the reader.
To make your paper more accessible you should explain what <s' \ s,s \ s'>
means in this context.

165: Can a segment be mapped to more than one skill dataset?

Humans learn with 1 or a few demonstrations.
This approach seems to need hundreds of demonstrations. Why?

The paper is good at presenting technical definitions and details. It would be even better if intuition was given, as well as definitions in words.

Words like operator, ground atom, and lifted atom seem to be necessary to understanding sections 3-5, but aren't explained until section 5.

How is this different from various other hierarchical approaches in
machine learning that simultaneously learn parameterized skills and/or
how to sequence them? Perhaps an explanation of how the components of
the AI planner relate to the components present in many other
hierarchical planning approaches would be beneficial.

What elements differentiate an AI planner from other hierarchical
approaches that, at a high-level, also plan in terms of discrete,
abstract skills? Intuitively, and in layman's terms, what do each of
the referenced components (operators, ground atoms, lifted atoms, add
effects, delete effects, etc.) of the AI planner do and how do they
fit into the system as a whole? Elaborating on these points prior to
and/or within section 2 would make it much easier to follow what all
the symbols mean and remember their significance throughout the rest
of the paper.

The term "operator" is not described in layman's terms.

Line 22: "We are motivated by cases where it is easier to design [] or learn [] symbols than it is
to design skills."  What are examples of these cases? It would be
helpful if the authors could elaborate on why it may be easier to
learn/design symbols than it is to design skills. For humans, many of
us can learn skills (riding a bicycle, throwing a pot on a potter's
wheel) for which we cannot easily articulate predicates (features we are
paying attention to) or how we do the task. Most bicycle riders are
unaware of the non-minimum phase nature of bicycle dynamics and
that you initially turn the front wheel in the opposite way you wish to
ultimately turn.

Line 112: What are the limits of add/delete effects? Do they limit
what can be done with an attribute and its value in any way?

Line 172 - 182: The concept of "lifting" is described in detail for
the first time after already being referenced multiple times.

Line 322: "To the best of our knowledge, our work is the first to
learn operators, samplers, and policies that are compatible with
search-then-sample TAMP [29, 30, 31]."  What makes the learned
operators, samplers, and policies compatible with search-then-sample
TAMP; and why has this not been the case in other works? To make this
claim, it should be clear what compatibility means and how it is
measured.

The result on the doors environment suggests that success it not possible without
the manually added motion planning skill. An ability to recognize where a planning
skill is needed, and even an ability to default to a explicit planner in this case would
strengthen the result.


robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are not well addressed
summary_of_recommendation: Ignoring philosophical differences with	the approach, I	think my
concerns focus on whether the evaluation is doomed to succeed.
The predicates are carefully chosen. Additional	symbolic
information is provided	by defining object (and value) types.
Not a lot of irrelevant	information has	been provided.
The simulations	have been greatly simplified. In this
kind of	work the scale or numbers of X (X = predicates,	objects, tasks,
demonstrations,	...) has to be large for the work to be
convincing.
The planning algorithm presented is clearly inefficient	and will
not scale well.

POST REBUTTAL COMMENT

I am still in favor of accepting this paper.

issues: Things that can be fixed in a reasonable amount of time:

WRITING
- improve and clarify writing (see my comments on that)
- make the paper stand alone more effectively, by moving some definitions and examples from the supplementary materials to the paper.

IN SIMULATION
- add large numbers of irrelevant predicates and operators
- see what happens if you don't filter out rarely used operators
- see what happens if you don't reject out of bounds subgoals
- wider variety of simulated tasks - ideally non-symbolic tasks and not effectively pick and place: paint something? repair something with glue, tape, or string? shape something such as clay or dough?

EFFICIENT PERFORMANCE
- explore several abstract plans in parallel rather than serially. This would deal with the issue of wasting a lot of time exhaustively sampling bad plans, and still allow adequate sampling on other plans.

EXTEND APPROACH
- start with demonstrations alone, measured with low level sensors (not manually defined high level
perceptions or predicates) as suggested in C.4.

Bigger changes I don't expect to be made, but could be made
- real robot implementation

MORE COMMENTS ON WRITING (to get around stupid character limits in the web interface):

Overloading lambda for object type and real valued features is confusing

In partitioning grouping segments if their affects are the same “up to object
substitution” is unclear. Does this mean any object of any type can be substituted into a
predicate
or does it mean only different object instances within the same type can be
substituted.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper is about learning skills in task and motion planning problems via demonstrations. Learning a skill involves learning a goal-conditional policy together with a sub-goal sampler, as well as an operator that defines the logical state transition (postconditions and effects) that the skill will induce, assuming some preconditions are true when the skill is invoked by the task and motion planner. 

The main contribution of the paper is showing how to learn policies, samplers, and operators from demonstration data that includes: the task definition (including a set of objects relevant for that task), the continuous state sequence, and the continuous controls. The continuous state transitions are segmented at contact points and thus split into skill dataset segments. These segments are lifted so that they are not grounded to particular objects, but types of objects. This is useful for learning general logical operators.       
strengths_and_weaknesses: The paper is comprehensive and generally well-written. Like many other related papers in this area, this paper loses some readability because of the heavy use of notation and definitions that are used to explain symbolic planning. This is unfortunately very hard to avoid, but I think the reader would benefit from a reduction of formal notation in the writing, if it is done in a way that increases intuition about the definitions. The use of a running example could help with this. Aside from writing, I would like to mention the following strengths (S) and weaknesses (W):

S1: The generalization results in the low-training data regime (a few hundred demonstrated trajectories) are interesting, especially as the number of object varies at test time.

S2: Long-horizon planning with learned skills seems like it works very well for TAMP, and this could definitely inform future design of such planners.

S3: Segmenting demonstration trajectories at contacts makes a lot of sense for pick-and-place tasks.  

W1: Lifting of demonstration trajectories that act on specific objects to more general types of objects seems error-prone, and I would have loved to see a more thorough analysis about how many logical state transition errors are introduced in the demonstration data via this pre-processing step. It seems brittle.

W2: The tasks selected for evaluation seem a bit simple, with only a few objects at training time (1-3) for many environments and (1-2) more at test time. I would loved to have seen more complex sequences of sub-goals involving more objects in the training environments.                  

          
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: My recommendation for a weak accept is based on the strengths of the paper and particularly S1, namely generalization of learned skills, including goal-conditioned policies and logical transition operators, with a few hundred demonstrations. It's an interesting result.   
issues: The issues that I would love to see fixed in the revision would be W1 and W2. They would both make the paper's analysis and conclusions more robust, and more convincing in terms of the reliability of the skill learning method.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: This work proposes a method for learning sets of pddl-style operators along with their associated low-level control policies from demonstrations. These "neuro-symbolic skills" can then be used in a straightforward bi-level search to solve challenging task and motion planning problems. The approach assumes knowledge of a set of predicates which are sufficient to define an abstraction of the low-level state, as well as a deterministic dynamics model for the environment. However, where traditional TAMP approaches also require hand-engineered operators, and associated control policies, this work shows how these can be learned directly from the given demonstrations. Experiments show that this approach can be data efficient, and produce skills which can generalize to new test problems, containing unseen numbers/placements of objects.

strengths_and_weaknesses: Strengths:
* The idea of learning operators which can be used with AI planning techniques *and* associated low-level control policies is promising. The approach can plan very long sequences of actions to achieve temporally extended goals.
* I appreciate the similarity of the approach to existing sampling-based TAMP methods.
* The domain-agnostic treatment facilitated by PDDL and object-centric state makes it easy to see how this method could be applied to new domains and tasks.
* In general, I appreciate the clarity with which the approach is presented. And in particular, the thorough analysis of experimental results. The many ablations that the authors included helped shed light on the impact of the different aspects of the approach, as well as the difficulties inherent in each evaluation domain.

Weaknesses:
* I think that the idea that certain skills might be easier to describe in terms of symbols (as opposed to a low-level policy/controller) is interesting. However, I felt that the experiments didn't showcase this, since every demonstration is generated by a hand-engineered policy. It would really help sell the point if there was an experiment where the skill was more difficult to engineer, and demonstrations were provided by humans.
* A related point is that all the of the training data are generated by systems that themselves use the same set of predicates and bi-level search. This seems a strong prior, and makes for very straightforward segmentation of training demonstrations. And the segmentation appears to be crucial for the success of the overall approach. It would be more convincing if the demos came from a human or some other system that has no knowledge of those predicates.
* The assumption of fully-observable state is very strong, but I understand the problem is already hard enough, and hope to see this method extended to the more challenging and realistic partial observability. 
* I think the choice of top-k planning seems like a weakness since the high level search for action skeletons isn't influenced by failures in the low-level policy. These failures should ideally help prune large portions of the high level search space.

robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I think the idea of using symbols as a scaffold for skill learning is interesting, and I like the fact that the learned operators allow these skills to be used by a standard PDDL planner to solve long-horizon tasks. The methodology described in this work is simple and clear, and the results are promising. Ideally, I would like to see this method used to learn from human demonstrations, but this is a good first step. 
issues: * I would like to see the performance of the learned operators compared to the hand engineered ones. I am curious whether the learned samplers would be more sample-efficient, given that they are trained on successful demonstrations.
review_title: null

Meta-review: metareview: Combining learning of symbolic high-level and operational low-level knowledge is an important and useful research direction, and bringing their learning together is a logical and potentially impactful idea. The paper proposes a sound method for this challenging task. The paper is well written and presented, although some explanation could be potentially simplified and required information is only found in the appendix. The presented results are very promising, especially wrt. to the generalization in the ‘low training data’ setup, and the provided ablations provide useful insights.

However, there are several concerns raised that should be considered and clarified. Mainly regarding clarification of some explanations and results, as well as the limitations of the assumptions and the clean/perfect simulations . See the following proposed action items based on the provided reviews (more details in the corresponding reviews).

Proposal of action items:
- Clarifying contribution wrt. to [11,12] 
- Clarifying generalization abilities wrt. to number of objects
- Discussing the importance of carefully hand-designed features/predicates
- Discussing what properties an abstraction must have to find successful solutions
- Discussing the sim-to-real gap, i.e., if the approach is (too) tailored to clean simulations
- Discussing the effect of KD2 on planning efficiency, and how to find really different abstract plans
- Consider moving important information from the appendix into the paper
- Consider used words and definition earlier, simplifying explanations
- Clarifying ‘compatibility’ wrt. TAMP
- Consider experiment(s) (or discussion) with more ‘noise’ (irrelevant predicates, objects, operators, noisy observations,..)

-----

I’d like to thank the reviewers and authors for the constructive discussions and valuable updates. The overall recommendations and discussions are in favor of accepting the paper, hence, my recommendation is to accept the paper. However, I agree with Reviewer Y3Ww that only changes in the paper really matter, and the authors are highly encouraged to include their answers and changes in the final updated manuscript.
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)