Learning Model Preconditions for Planning with Multiple Models
Authors: Alex Licari LaGrassa
Oliver Kroemer
Venue: CoRL2021 Poster (2021)
Keywords: manipulation
planning

Abstract: Different models can provide differing levels of fidelity when a robot is planning. Analytical models are often fast to evaluate but only work in limited ranges of conditions. Meanwhile, physics simulators are effective at modeling complex interactions between objects but are typically more computationally expensive. Learning when to switch between the various models can greatly improve the speed of planning and task success reliability. In this work, we learn model deviation estimators (MDEs) to predict the error between real-world states and the states outputted by transition models. MDEs can be used to define a model precondition that describes which transitions are accurately modeled. We then propose a planner that uses the learned model preconditions to switch between various models in order to use models in conditions where they are accurate, prioritizing faster models when possible. We evaluate our method on two real-world tasks: placing a rod into a box and placing a rod into a closed drawer.

Learning Model Preconditions for Planning with Multiple Models
Alex LaGrassa Roboties Institute
Oliver Kroemer Roboties Institute Carnegie Mellon University okr oxreer@aaddew cmu, edu
Carnegie Mellon University cmu. edu
cmu.
Different models can provide differing levels of fidelity when a robot is planning. Analytical models are often fast to evaluate but only work in limited ranges of conditions. Meanwhile, physics simulators are effective at modeling complex interactions between objects but typically more computationally pensive. Learning when to switch between the various models can greatly improve the speed of planning and task success reliability. In this work, learn model deviation estimators (MDEs) to predict the eтror between real-world states and the states outputted by transition models. MDEs be used to define model precondition that deseribes which transitions are accurately modeled. We then propose planner that uses the learned model preconditions switch between various models in order to use models in conditions where they are accurate, pri- oritizing faster models when possible. We evaluate method tasks: placing rod into box and placing a rod into a closed drawer.
Abstract:
real-world
Keywords: planning, manipulation
Introduction
Predictive models that are helpful for intelligent behaviors can take a variety different forms, e.g. analytical models [1, 2], physics simulations [3, 4]. and learned models [5, 6, 7], and the best choice of model is context-dependent. For example, a high fidelity simulator can model complex interactions between large number of objects when dumping pile of non-convex objects table [8]. However, for a simple tabletop pick and place task, simulator model often unnecessary when simple kinematic model is available.
State
Skill (8,2) pro(M) (s, pro(M,) (s pre(M.) Model to use
9,"
M,
M,
M, Low-sesolution physics simulator
8,"
X
X
M,
kinematic model
M,
Simulator
Hiph-resolution
Ma(sya,)
M
physics simulator
Syn
Analytical model
M,(s,a,)
Figure 1: Each colored region on the left represents a model precondition, which we learn in this work for analytical and simulator models for manipulation skills. In this example, kinematic model is faster than low-lidelity simulator, and low-fidelity simulator is faster than high-fidelity simulator. On the right we show for each step which stale-action pair (ft,af) was the precondition for each model, and best model to use to evaluate that edge. The states and actions on the path chosen for the plan have a bold border. States not expanded because of high model cost have dashed border. Multiple actions at a state $; are denoted by superscripts: For example, two different actions from So would be [a8, ao).
5th Conference on Robot Learning (CoRI. 2021), London, UK.

---

A planner may need to make many predictions to find plan, so it is important to only use com- putationally expensive models when they are most needed. We define model preconditions, shown abstractly in Figure 1, that demarcate regions of states and actions where the model accurately rep- resents real-world dynamics. We use those model preconditions during planning to leverage the complementary advantages and disadvantages of different models.
To capture the model preconditions, the robot learns to predict the total state deviation as a con- tinuous value for each model. These deviation predictions are used to inform the planner to avoid transitions that are inaccurately modeled. The amount of acceptable model error is task-dependent, but by predicting state deviation as continuous value rather than a binary one, the thresholds can be set based on the task accuracy requirements.
This work's contributions include a method for defining model preconditions using a learned scalar model deviation term and then using those preconditions for planning with multiple models to min- imize model error at execution time. We first show how to fit Model Deviation Estimators (MDEs) from planning-relevant data to predict the error between a transition model and real-world dynam- ics. We then use MDEs to define the model preconditions for multiple skills for specific tasks. We evaluate our method for learning and planning with MDEs on two real-world tasks: placing rod that can rotate in-hand into a box and placing a rod into a closed drawer. The robot has access to incomplete analytical models and simulator model for each action class. Like in many real-worth planning applications, none of the models match the real-world dynamics for all possible interac- tions. We compare two planning approaches with MDEs against other model selection baselines.
2 Related work
We review related work relevant to our two main contributions: predicting model error from data and planning using multiple imperfect models by reasoning about planning speed and reliability tradeoffs.
Improving planning reliability with past experience: Learning model preconditions for planning draws from the high-level idea of identifying similarities to states encountered in previous plans to apply behaviors that previously led to successful plans, which can significantly improve planning speed [9, 10J. Plan reliability can also be improved by learning skill preconditions, which describe the states from which skills achieve desired effects with high probability [11, 12, 13]. Instead of using skill preconditions, we use transition model preconditions.
Avoiding regions with model error: For predicting where the model is accurate using data, Chou et al. [14] and Knuth et al. [15] use hyperspheres to represent proximity to training data for learned model as a proxy for the model precondition. CMAX and CMAX++ also use hyperspheres, but use points where the action effects differed significantly from those predicted by the internal model [16, 17]. However, proximity to training data and predicted collisions do not always capture model accuracy when executed on a real robot. Our work proposes a method to fit an estimator from plan execution data to predict whether model error will occur. Other works bias search away from states where the simple planning model is known to be inaccurate, but also do not use other models to compensate for transitions the simple model cannot model accurately, while our planner uses a slower model where appropriate [15, 14, 16, 18). This distinction of not using alternative models when model error predicted especially significant when no one model can be used to compute a plan accurately, such as when each model only reasons about a subset of interactions necessary to achieve a task.
Predicting model error: The works most similar to ours use visual similarity to model the un- certainty of their simple model in Power and Berenson [19] and learning a classifier in the work of Mitrano et al. [20] and McConachie et al. [18]. The primary difference is that they do not use their model error estimates as criteria for switching models, whereas we use the planning data to learn model error estimates to decide between multiple models. McConachie and Berenson [21] does use multiple models, but uses task progress rather than error as selection criteria and does not do multi-step planning. Additionally, these works assume access to a simulator accurate enough to compute the ground-truth labels for accurate model predictions. We relax that assumption by treating the simulator as additional model, which can also exhibit model error. Our labels then come from the real world. The methods above are also focused on a rope manipulation domain, where errors can usually be corrected with a recovery policy, as shown in [20]. Many actions in the
2

---

manipulation domain in which we evaluate our model error prediction lead to unrecoverable states, such as an object lying in an un-graspable pose in a constrained space.
Planning with multiple models: The work of Saleem and Likhachev [22] uses a rule where a simulator is only used when collision is detected using internal model to speed up planning. However, there are many situations where there are transitions in which objects are in contact but an analytical model can still be used. Our planning algorithm is based on Multi-Graph Multi-Heuristic A* [23], which presents an algorithm that selects between models which operate on different parts of the state. The models we consider in this work operate on the same objects in the state, which makes choosing model not straightforward. Furthermore, for the models we use, the best model to use depends on both the state and action rather than just state.
3 Problem Formulation
The overall problem is to first learn the MDEs to predict the total model deviation given state, ac- tion, and transition model, then use MDEs for planning using multiple models, prioritizing planning using the faster models where possible while still minimizing cost.
The state space is denoted by S and is assumed to be fully observable at the end of an action. The ac- tion space A is set of closed-loop skills A. parameterized by 0 which are available to the robot. A user-defined skill-specific parameter generation function generates potentially useful parameters given state. Skills follow the options formulation [24]. Skills are executed using low-level con- troller until a termination condition is reached. The actions are intended to be run for a significant period of time such that feedback control at a lower level of abstraction
correct for low-level errors, leaving the MDEs to capture model error that is harder to correct. Skills have an associated precondition set, pre(a) C S describing the set of states where the skill can be executed. The skill preconditions narrow the space where the skill has defined behavior, but do not comprehensively describe the set of states where the dynamics models are accurate. The skill precondition simply re- lates to the skill's 's upplicability and are separate from MDEs and models of the skills' effects, which we describe next.
The robot is given an ordered list of models Mo, M Mx]. used to compute the forward dy- namies: 8' t (s,s, a). The models are ordered by increasing computation speed, where M; is slower to evaluate than M, if i < j. For example, M, may be a high-fidelity simulator that slow but very accurate M, simulator with coarse timestep discretization, M2 an analytical model requiring more computation, and M3 simple linear model. It is preferable to use the model with the highest i, corresponding to the fastest model, that is accurate for that (s, a). To measure model accuracy, the robot is given a distance function between states d(& 85). such as Euclidean distance.
The MDE predicts d from dss) and a, where ((s) is function that extracts relevant features such as object poses or distances between objects.
Lastly, the robot is given a task that includes a goal set, cost function, and optionally additional task-specific action parameter generators. The state satisfies the goal if it is in the goal set S9. The cost function describes the cost of high-level state transitions and is denoted by c(s, a, 8').
To train the MDEs, the robot needs first collect an offline dataset D of real-world transitions (s, a, s'). We assume the robot knows the initial state distribution but not all states visited during planning. Initial states are sampled from the initial state distribution, but the rest of the states are determined from planning. Then the robot trains the MDE d(ss, a) which estimates d(s', using D. The eventual goal is for the robot to search for high-level plan of actions (do, 01,..., -11 to create a sequence of states [so, 81, .1....,877 ST] such that the final state is a goal state: 8T E S9.
4 Approach
First we explain how to define and learn model preconditions using MDEs. We then show how MDEs can be used in planning to use multiple models while prioritizing states that can be reliably evaluated using simpler models.
4.1 Learning Model Preconditions Using Model Deviation Estimators
If a transition is in a model's precondition, then that model can be used to accurately model that transition. model precondition defined using MDEs as:
pre(M.) = (sx,a) d(,,(s,,), < dmax
(1)
3

---

Training data for MDEs is collected by computing and executing plans using all models. If a path the goal is found, it executed. Each transition is saved as (s, s') tuples for every transition observed in the path. The list of transitions is expensive to collect as it requires real-world data, but several datasets for MDEs can be derived from it.
To train the MDE for particular model M, s' is computed using M, for each (s,a) in D. The labels are d(s', 8'). Training data for transitions is used across all models even though that transition was computed using just one model during planning.
The inputs are state features, d(s) concatenated with A. o(s) can be the identity function, but local features such as the distances between objects can enable easier generalization.
We train an MDE for every combination of skill and model, though MDEs can be shared if skill parameters have similar meaning. The reason why MDEs are regression models instead of classifi- cation models is to enable sharing of MDEs across tasks with different accuracy requirements. To model the MDEs we use a 3-layer MIP with 32 hidden units in each layer.
Next we describe the loss function for MDEs. An underestimated predicted deviation is worse for plan reliability than an overestimated predicted deviation. Overestimates cause the planner to be overly conservative, but an underestimate can cause plans to be executed unreliably on the real robot, As a result, we propose a loss function that penalizes underestimates more than overestimates: Ty (d, d)- = ei max(00, d d)² + 02max (0, d d)². To get the desired behavior, we set C1 > C2.
4.2 Planning With Multiple Models Using MDEs
Model preconditions can be used in the same manner skill preconditions by using the model preconditions as a constraint during planning. such as by using Equation as an inequality constraint during optimization, where the M used at each t are additional variables. Model preconditions as predicate constraints can be applied to STRIPS-style planners using actions with a particular M to compute skill effects by including (s, E pre(M) to the existing action preconditions as an additional predicate.
Although MDEs can be used to select between models in more general setting, we propose planner that can more effectively use MDEs by using a user-defined model priority weighting to prioritize expanding subgraphs that are less expensive to evaluate. Our planner prioritizes simpler models by using a weighting factor inspired by the one used in Multi-Hcuristic A*(MHA*) [25]. Because our planner uses successors from different models, we also build on Multi-Heuristic Multi- Representation A [23].
Our proposed planning algorithm uses up to K models for each skill. Models can be shared across skills. The planner expands K implicit bidirectional graphs, which maintain their own open queues for nodes to expand and closed sets for nodes that have already been expanded, but all graphs share successors. All open queues are initialized with the start state and all closed sets are initialized to D.
An anchor search expands its graph by selecting between all models, and the K additional graphs expand using the K faster models. Priorities between models are adjustable using a list of model preference weights, w [wo, W1,... WK-1] decreasing in value representing how much to penalize slower models. We now describe how the separate queues and graphs are used to switch between models.
Similar to standard Weighted A (WA*), each planning node has cost g, heuristic value h, and priority f = + eh. The function MINKEY() returns the node in a queue with the lowest f value. To handle partial expansions, node can optionally have set of un-evaluated actions associated with it. (s). The anchor search uses open queue OPEN, and closed set CLOSED,- It uses a full expansion, which computes the successor with the fastest model that satisfies the precondition for a particular state and action. If no model satisfies the precondition, there is no successor for that transition. Nodes that are fully expanded in the anchor search are added to all closed sets. Each additional search maintains its own open queue OPEN, and closed set CLOSED; It computes successors using a partial expansion only using M, as long as (s, a) E pro(M,), saving other actions for expansion by other graphs.
Graph and node selection: We expand from queue i using a rule inspired from MHA*:
i arg min w; icK

---

(a. empty
[=0
generate x' 'ussing fastest model st. (t. a) prark)
statc:
graph:
is gool
generate
A
Full expansion
(9) non-empty
66 -8
state
generate using M,
1>0
(6.a) i preM
return path
t0 4 Partial expamion
Figure 2: Graphical representation of the two types of expansions using model preconditions. Green ovals are data and yellow rectangles represent operations on data. The left side shows how actions selected for both expansion types. The right side shows full expansion (top) and partial expansion (bottom).
If O, then node is expanded in the anchor search using the full expansion. If >> O then the node is expanded using a partial expansion for graph i. The node to be expanded is MINN,, as with standard WA* # search.
We begin by describing steps used for both types of expansions, and then describe what is different. All expansions begin by first checking the goal condition. As shown in Figure 2, the search is terminated when goal state is expanded. Otherwise, expansion begins by selecting the set of actions to evaluate, A. A is Aine if Aine is non-empty. If not, parameter generation function generates a candidate list of possible parameterized actions. The graph to expand chosen using the rule in Equation 2. For each successor s', if the path found to s' is lower cost than any current path found to s' in any graph, the path is updated using s'. Then s' is added to to all OPEN; if is not in CLOSED;.
Full expansion: Full expansion used as a fallback when the other searches with faster models run out of nodes or only contain highly suboptimal nodes. As shown in the top right box in Figure 2, full expansion evaluates all successors using (s,s,) a) for the highest value of that satisfies (s.a) E (M.). Then, is added to all closed sets.
Partial expansion: The intuition behind using partial expansion to evaluate nodes using faster models sooner while delaying slower evaluations. For all (s, a) E pre(M;), the successors s' are computed using M,, and they are added to all open queues. For all states and actions not in the precondition, the successors are not evaluated. Instead, those actions are added to Ame(s) for later evaluation in other graphs. then the node is added to CLOSED,. This step helps bias the search to nodes that are cheap to evaluate, while still allowing the planner to reason about when to use more expensive models for completeness or optimality.
5 Experiments
First, we describe the metrics we use to evaluate our method, then describe the experimental setup use for real-world task domains shown in Figure 3: placing one of two steel rods in a box (RodInBox), and in a drawer (RodInDrawer). Then, we show the accuracy of MDEs for each skill. Finally, we evaluate two planners that use MDEs on two real- world tasks against baselines: one using a single queue with model preconditions as constraints and another using our multiple-qucue planner described in Section 4.2.
We compare timing metrics, demonstrate the model selection for each skill in the RodInDrawer task, and discuss how the model selection distribution in each algorithm impacts plan time and reliability. For plan performance, we test: /) the time to compute plan (seconds), 2) the model evaluation rate (model evaluations per second) to measure the planner's ability explore efficiently, 3) the success rate in computing a plan, and 4) the success rate achieving the goal in the real-world tasks if a plan was found (to measure reliability).
For both RodInBox and RodInDrawer tasks, a Franka Emika Panda manipulates one two steel rods into a desired container. The rod to place in the container is chosen randomly. Due to its weight, the rod can drop or rotate in the gripper (Figure 6). Both rods are placed in arbitrary reachable poses not occluded by other objects on the table.
Tasks:
The first task RodInBox. A state is in S if the target rod is in the green box shown Figure 3. The purpose of this task is to evaluate what happens when our method is applied on task where high precision not necessary for task completion, but the effects of some actions are easier to model than others. The second task we evaluate on is RodInDrawer, shown in Figure 3 where the
5

---

(a) RodInBox (Sim)
(b) RodInBox (Real)
(c) InDrazer (Sim)
(d) RodDawae (Real)
Figure 3: Robot setup for test tasks in simulation and on the real robot.
goal is to place a rod in the partially closed drawer. Goal states are those where the rod within the bounds of the drawer. Heuristics for both tasks are in Appendix 1.
Actions are movements in Cartesian end-effector space, implemented in the real world using the library from Zhang et al. [26] and with end-effector attractors in simulation. In simulation, the world is fully observable. In the real world, the rod poses are estimated using an the overhead camera. When a grasped object is occluded by the robot, an in-hand camera is used to confirm that the object is in the gripper, and the previously recorded pose is used. This perception sufficient for our use case because the skills with the most variability do not end with an object in the gripper.
Skills: The skills the robot uses are Pick, LifiAndDrop. and OpenDrawer. Because the effect models used are high-level and the end-effector moves horizontally at a height above all obstacles, collisions are only checked the beginning and end of each motion. The state includes the pose of all objects in the scene, including the robot end-effector pose. Pick is parameterized with the goal pose, which sampled around the target rod. LifiAndDrop brings the end-effectorr and rod to a target location, then opens the grippers. The precondition of LiftAnddrop is that an object is between the grippers and that the goal pose will not cause collisions. OpenDrawer moves the gripper location relative to the front of the drawer, then opens the drawer moving to a final pose. Detailed descriptions of all the skills, including hyperparameters, are in Appendix 7.1.
fixed
The robot has access to simulator and two analytical models. One analytical model, Analytical (Pick & Place), only computes the transition model for pick and place actions, assuming a rod close to the gripper is rigidly attached. When the rod is placed, it falls to height determined by the object (if any) directly below it. Another model, Analytical (Drawer), only reasons about a simplified articulation mechanism of the drawer, assuming the drawer edge is rigidly attached to the gripper if both surfaces are close, but does not model interactions between the rods and drawer. Details about each analytical model are in Appendix 7.2.2
Models:
The simulator use for both tasks is IsaacGym [3]. For RodInBox, shown in Figure 3(a). The simulation parameters used for both experiments are shown in Appendix 1. For IDrawer, the drawer is modeled Figure 3(c). To evaluate the result of a skill from a state 8, the simulator sets the state of the world to 8, executes skill a and returns the resulting state 8'
setup with box
single drawer chest using cuboids, as shown in
5.1 MDE Accuracy
In this section, we evaluate MDE ac- curacy on each skill used, which cor- relates with the variance of d. As shown in Table 1, deviation predic- tion error is highest for LiftAndDrop because there is a wider range of pos- sible outcomes. The predicted devi- ation error for the simulator model low but not zero because predict- ing the precise deviation requires an accurate
Skill Pick Pick
Model
MAE
d
Analytical (Pick & Place)
0.2
0.4 (0.4)
Analytical (Drawer)
0.2
0.4 (0.4)
Pick
Simulator
0.7
4.7 (9.5)
Lioo
Analytical
(aciical(PPPPPe (Pick & Place)
I.9
(..4D
LifiAndrrop
Analytical (Drawer)
7.6
11.6 (8.0)
LiftAndDrop
Simulator
2.5
11.0 (10.7)
OpenDrawer
Analytical ( (Pick & Place)
0.4
16.0
OpenDrawer OpenDrawer
Analytical (Drawer)
0.6
2.1 (0.3)
Simulator
1.2
7.0 (0.5)
For example, a dynamics model. rod may drop in simulation earlier than it would on the real-robot system by dragging on the ground differently. Since Analytical (Drawer) does not model movement of the rods, the exact deviation de- pends on the state and parameters, but the error always predicted to be high for LifiAndDrop, which is sufficient information to guide planning. The deviation for Pick using Analytical (Drawer) is low because modeling Pick does not require any physical interactions between robots and rods, and the robot is not in contact with the drawer. Simulator deviation prediction error for Pick is high be-
Table 1: Mean absolute error (MAE) and d (mean and standard deviation) in centimeters for each model/skill combination.

---

Method
Plan time
Model eval.
Method
Plan
Model eval.
per second
time
per second
Ours multiple queues
0.40
25.2
Ours multiple queues
3.69
19.0
Ours one queue
8.98
1.13
Ours one queue
5.27
6.0
Random
40.00
0.25
Random
44,36
0.6
Analytical (Pick & Place)only Simulator only
0.19
113.58
Simulator only
103.34
0.5
31.23
0.5
(a) RodInBox
(b) RodInDrawer
Table 2: Planning times for methods that can find plans. We show the average time to compute a plan (not. including timeouts).
cause when a drawer is in the environment, the joints sometimes interact with the drawer, causing interactions that do not occur in the real world.
5.2 Planning Speed
For planning, we evaluate the performance from ten different initial start states per task. RodInBox uses Pick and LifiAndDrop, sampling five parameter vectors per skill. RodInDrawer uses Pick, LiftAndDrop, and OpenDrawer, with three parameters generated per skill. = 5 for RodInBox and [Simulator,
10 for RodInDrawer. The model preference weights are [10,1] corresponding Analytical (Pick & Place) for RodInDrawer and [10,1.1.1] corresponding to [Simulator, Analytical (Drawer). Analytical (Pick & Place) ] for RodInBox. The cost function given by the total end effector distance covered in the trajectory. For timing experiments shown in Table 2a, we allow the planner to run for at most 300 seconds (5 minutes).
We compare planning using MDEs to planning with each individual model, and ran- domly selecting which model to when computing each successor. Additionally, we perform an ablation test by removing the additional queues and only performing full expansions using the anchor search, though still evaluating the fastest model that satisfies the preconditions.
Baselines:
For RodInBox, (Table 2a) the planning time and model evaluation rate are fastest using only the analytical model. Random model selection and simulator-only planning are the slowest. Because our method uses the fastest reliable model wherever possible, and for this task a short plan to the goal can be computed using only the fastest model, the model evaluation rate for our method is still high. The one-queue planner that uses MDEs is slower because the model precondition for the simulator model for LiftAndDrop is usually satisfied, so some actions that unnecessary to find plan are simulated.
The planning speeds for both methods using MDEs in RodInDrawer (Table 2h) are faster than the simulator-only baseline because it can always use an analytical model for Pick and OpenDrawer, and sometimes use the analytical model for LiftAndDrop. The improvement of using multiple queues in this domain is smaller than for RodInBox, which we discuss further in the next paragraph. The random baseline is the slowest because it uses the largest number of simulator calls.
Both MDEs and the planning method can affect the distribution of models chosen. For example, Figure 4 (left) shows planning with one queue causes the planner to compute successors using tran- sitions in the model precondition of more expensive models the same priority of sets in the faster models, which causes some actions to be simulated that would not be if they were using our multiple- queue planner, causing the longer planning times for RodInBox. For RodInDrawer, the one-queue ablation test selects very similar models to the multiple-queus planner, so using a single queue does not significantly affect planning speed. The reason that the simulator model more inaccurate in RodInDrawer awer than in RodInBox. The MDE can detect the task indirectly since the parameter distri- butions for the two tasks are different. As a result, the one-queue planner rarely uses the simulator, which can also be seen in Figure 4 (right).
5.3 Reliability of Computed Plans
The experiments shown in Figure 5, evaluate the ability of our method to improve plan reliability without significantly increasing planning time by balaneing model accuracy and computational cost.
7

---

Simusator
20
Analysicll
Aralycical (Drawer)
UiSAndrrop
Pck
Lp
Pc
LitAndOrop
OpenDrawer
LitAndOrop
OpenDransr
Pick
LitAnsDrap
OpenDrower
Pick
LitAnsDres
Sail
Figure 4: Average number of models evaluation of each type for all three skills for RodInBox (left) and
RodInDrawerrrighul across methods that use multiple models
1.00
1.001.001.00
1001000.000
001.000..00
1,001.00
Ours: queue
eecodto dien eucceds
1.00
0.8g0:90
Pan tnntouns
0.50
Ours mutiple queues
0.75
Random
0.40
Analytical Hace
Simurator
0:40
0.25
(Drawer)
0.25
0.00
RedInBox
0.00
RadIndrawer
0.000.00
0.00
RedInBox
$..00
RadIndrawer
0.000.00
Figure Success rate finding plans (left) and found, executing them until goal (right)
(a) No pivor (Analytical)
(b) Drop during transport (Real)
(c) Hitting target container (Real)
(d) Unrealistic physics (Sim)
Figure 6: Situations where planning models are inaccurate. (a) Analytical (Pick & Place) assumes rigid attach- ment, although this (b) causes the rod to either drop during transport or (c) rotate and hit the target container. (d) The simulator sometimes exhibits unrealistic physics such as the rod sliding over the box walls.
All methods that use a model that reasons about interactions between the rod and robot are able to compute a plan the allotted time for RodInBox. the more challenging RodInDrawer task, the simulator-only baseline and one-queue version of our method sometimes fail to find a plan within the time limit. Plan success is highest for both methods using model preconditions and the simulator- only baseline. MDEs cause the planner only transitions where the model is predicted to be accurate, such as using LiftAndDrop when the rod is grasped in the center. The simulator is more accurate than the analytical models plans using it often succeed. In contrast, plans found using the analytical-mont pooeeal-ooel-onlyy and andom-model baselines often fail because the lowest cost paths often include grasping the end of rod, which is predicted to be rigidly attached by the analytical model (Figure 6a). In the real world, grasping the end of the rod causes it to pivot, which usually causes the rod to drop during transport (Figure 6b) after hitting the target container (Figure 6c). Although the simulator can model pivoting in the gripper, unrealistic dynamics sometimes occur (Figure 6d).
6 Conclusion
We present a method for defining model preconditions using MDEs that predict model deviation given a state and a parameterized skill, which can be used inform tradeoffs between models when each has complementary advantages and disadvantages in evaluation time and accuracy. Although MDEs can be used as constraints, we show planner different models that prioritizes planning using the faster models. Experimental evaluations show a speed up in planning by choosing between multiple models while keeping high plan reliability during execution. In both evaluation tasks, results show that robot can learn and reason about which transitions need to use more expensive models, which transitions none the models can eval- uate accurately, and how to inform planning with that information. For placing rod in drawer, show that the robot can combine two low-fidelity models intended for representing different interac- tions, while avoiding using expensive simulator that models all interactions. For future work, will modify our framework to share more data across skills by using a shared action representation, which will also allow changing low-level model multiple times during skill execution. Further- more, we will evaluate our method on tasks where model selection more difficult, such as liquid, paper, and other deformable object manipulation.
uses multiple queues corresponding to

---

Acknowledgements
This work was supported by the Office of Naval Research Grant No. N00014-18-1-277, Army Research Laboratory grant Foundation Grant No. CMMI-1925 and IIS- 1956163. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of our sponsors. The authors also thank Kevin Zhang for assistance with robot experi- ments, well Shivam Vats, Jacky Liang, Mohit Sharma, and Saumya Saxena for their support and discussions.
part of the A212 program, and National Science
References
[1] S. Goyal, A. Ruina, and J. Papadopoulos. Planar sliding with dry friction part 1. limit surface and moment function. Wear (Amsterdam, Netherlands), 143(2):307-330, 1991.
[2] Z. Pan and K. Hauser. Decision making in joint push-grasp action space for large-scale object sorting. arXiv preprint arXiv:2010. 1aa0000000000 10064, 2020.
[3] Liang, V. Makoviychuk, A. Handa, N. Chentanez, M. Macklin, and D. Fox. GPU-accelerated robotic simulation for distributed reinforcement learning. In Conference on Robot Learning, pages 270-282. PMLR, 2018.
[4] E. Coumans and Y. Bai. PyBullet, a python module for physics simulation for games, robotics and machine learning. http://pybullet. org, 2016-2021.
[5] D. Hafner, T. Lillicrap, Fischer, R. Villegas, D. Ha, H. Lee, and J. Davidson. Learning latent dynamics for planning from pixels. In International Conference on Machine Learning, pages 2555 2565.. PMLR, 2019.
[6] B. Sundaralingam and T. Hermans. In-hand object-dynamics inference using tactile fingertips. IEEE Transactions on Roboties, 2021.
[7] A. Nagabandi, K. Konolige, S. Levine, and V. Kumar. Deep dynamics models for learning dexterous manipulation. In Conference on Robot Learning, pages 1101-1112. PMLR, 2020.
[8] M. Macklin, M. Müller, N. Chentanez, and T.-Y. Kim. Unified particle physics for real-time applications. ACM Transactions Graphics (TOG), 1314):1-12,2011 2014.
[9] B. Kim, Z. Wang, L. P. Kaelbling, and Lozano- Learning to guide task and motion planning using score-space representation. The International Journal of Robotics Research, 38 (7):793-88) 812, 2019.
[10] M. Phillips, B. J. Cohen, S. Chitta, and M. Likhachev. E-graphs: Bootstrapping planning with experience graphs. In Robotics: Science and Systems, volume 5, page 110, 2012.
[11] G. Konidaris, L. P. Kaelbling, and T. Lozano-Perez From skills to symbols: Learning symbolic representations for abstract high-level planning. Journal of Artificial Intelligence Research, 61: 215-289, 2018.
[12) M. Sharma and O. Kroemer. Relational learning for skill preconditions. In Proceedings of (CoRL) Conference on Robot Learning, November 2020.
[13] Z. Wang, C. R. Garrett, L. P. Kaelbling, and T. Lozano-Pérez. Learning compositional models of robot skills for task and motion planning. The International Journal of Robotics Research, (964 2021.
[14] G. Chou, N. Ozay, and D. Berenson. Model error propagation via learned contraction metrics for safe feedback motion planning of unknown systems. ArXiv, abs/2104.08695, 2021.
[15] C. Knuth, G. Chou, N. Ozay, and D. Berenson. Planning with learned dynamics: Probabilistic guarantees on safety and reachability via Lipschitz constants. IEFE Robotics and Automation Letters, 6(3):5129-511) 2021.

---

[16] A. Vemula, Y. Oza, J. A. Bagnell, and M. Likhachev. Planning and execution using inaccurate models with provable guarantees. In Proceedings of Robotics: Science and Systems (RSS '20), July 2020.
[17] A. Vemula, J. A. Bagnell, and M. Likhachev. CMAX++ Leveraging experience for planning and execution using inaccurate models. arXiv preprint (XX.::2009.0994222 2020.
[18] D. McConachie, T. Power, P. Mitrano, and D. Berenson. Learning when to trust a dynamics model for planning in reduced state spaces. IEEE Robotics and Automation Letters, 5(2): 3540-3547, 2020.
[19] T. Power and D. Berenson. Keep it simple: Data-efficient learning for controlling complex systems with simple models. IEEE Robotics and Automation Letters, 184)-19841 2021.
[20] Mitrano, McConachie, and Berenson. Learning where to trust unreliable models in an unstructured world for deformable object manipulation. Science Robotics, 6(54), 2021. URL https://robotics. wcw.eieccsccceceeeeeoeeeoeae org/ congen/n
7144/ebbd170.
[21] D. McConachie and D. Berenson. Bandit-Based Model Selection for Deformable Object Manipulation, pages 704-719. Springer International Publishing, Cham, 2020. ISBN 978- 3-030-43089-4. 111111..00777----000000
45. URL https::/ddo.orrg/11
[22] M. S. Saleem and M. Likhachev. Planning with selective physics-based simulation for ma- nipulation among movable objects. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 6752-6758. IEEE, 2020.
[23] D. Youakim, A. Dornbush, M. Likhachev, and P. Ridao. Motion planning for underwater mobile manipulator by exploiting loose coupling. In 2018 IEEF/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7164-7171. IEEE, 2018.
[24] R. S. Sutton, D. Precup, and S. Singh. Between MDPs and semi-MDPs: A framework for tem- poral abstraction in reinforcement learning. Artificial intelligence, 18:1-22):1121111 1999.
[25] S. Aine, Swaminathan, V. Narayanan, V. Hwang, and M. Likhachev. Multi-heuristic A The International Journal of Robotics Research, 35(1-3):224-243, 2016.
[26] K. Zhang, M. Sharma, Liang, and O. Kroemer. modular robotic arm control stack for research: Franka-interface and frankapy, arXiv preprint arXiv:2011.023398, 2020.
[27] C. R. Garrett. Pybullet planning, 2020.
URL https://pypi: pwog//t
pybullet-plaminng
[28] D. P. Kingma and Ba. Adam: A method for stochastic optimization. In Y. Bengio and
Y. LeCun, editors, 3rd International Conference Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http: // //arxiv.org org/ abs httgg/ab//41.....eg 1412.6980.
10


Reviews:
[{"review_id": "zqkC0W2uQ4c", "reviewer": "Reviewer_MmFq", "summary": "This paper presents a method for planning robot manipulation tasks using multiple forward models of different fidelities. The authors propose learning to predict the expected total error of a model's prediction for a given state and action. The authors then propose a planning method which simultaneously rejects actions which have high predicted error and biases the search towards using lower fidelity models. The approach is demonstrated on two experiments each with two models, showing improvements in planning speed over using only a high fidelity model, while remaining competitive in the success rate of the generated plans. ", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n-The paper is well motivated, planning with multiple imperfect models is an area of great interest to robotics. \n\n-The planning method utilizing different models seems quite a sound idea. It is also an interesting idea that we might want to preferentially use different models for different situations. \n\n-Experiments demonstrate that the method is useful for speeding up planning without compromising performance on a real robot with a very small dataset. \n\nWeaknesses\n-The main weakness of this paper is the clarity of writing and presentation. The paper would benefit from thorough revisions and editing for clarity in all sections. A few examples are described in the “Issues” below.\n\n-The insight that model error underestimation may be better than overestimation is quite interesting and the proposed solution is straightforward. However, it is not clear the method which uses this assumption actually improves performance. Experiments are necessary to verify that underestimation is indeed better.\n\n-The experimental results would benefit from an ablation. For instance, it is unclear why the second queue, which only expands using the simplest model, is necessary. The first queue already preferentially expands using the simplest model which satisfies the precondition, so does this not provide enough of a bias toward using the simple model?\n\n-The author's planning framework could very easily incorporate using a binary classifier to predict if the action is in the precondition of the model. The authors claim their method of predicting total state deviation should be ‘more transferable across tasks’ (line 28), however this claim is unsubstantiated.\n\n-I do not think the discussion and comparison of the different models (GP vs MLP etc.) used to learn the MDEs to be particularly useful or relevant to the main contributions of the paper. \n\n-The tasks used for the experiments are very simplistic and very simple models of pick, place, and drawer-slide actions can easily be used to accomplish them. In fact, such tasks have been done with kinematic motion planners for several decades. A task where the robot is manipulating an object that has interesting dynamics (which actually requires a high-fidelity simulator, at least sometimes) would show the benefit of this method much more clearly.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Overall the idea of using multiple models to efficiently find plans for manipulation tasks is interesting and the approach has some technical novelty. However, the tasks used in the experiments are very simple and can easily be done using traditional pick/place/open-drawer actions generated by existing planners with very simple models, thus the usefulness of this method over state-of-the-art is not clear. Also, the paper is very hard to read and the clarity needs to be significantly improved.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Poor: Difficult to follow and understand.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "Very good: Comprehensive knowledge of the area"}, "extra_text": {"issues": "-The caption of figure 1 describes a figure on the left which does not appear to exist\n\n-Several points in the paper, the notation switches from M0 being the high fidelity model and MK being the low fidelity model  to the other way around. (For example figure 1 vs line 96 vs line 177). This makes it very difficult in section 4.3 to understand exactly which model is being used when, I am left having to guess the author's intentions. \n\n-I had a great deal of difficulty understanding section 4.3, particularly the partial expansion from open 1. Some sort of graphical explanation / a figure or algorithm might be very useful for getting the information across. \n\n-Videos could use some work. First caption of the planning explanation video is very difficult to parse for instance. \n\n-For the videos of the real robot experiments, the video for the first experiment doesn’t seem to complete, I am unsure what to make of it. The Video for the second experiment doesn’t include baselines. If time is an issue I would recommend changing the video speed for the experiments to be 4x real time (or something). \n\n-The fact that there is always a high fidelity model to fall back on means that in the worst case it can likely find a good plan albeit slowly. The results for the drawer experiment showing that the analytical model can be used even when it completely fails to account for the articulated object is very interesting. Could this framework be used if you instead had two low fidelity models with different limitations? This would be really interesting to see. \n\n-What happens when you use your framework using only 1 model? This should only expand nodes for which the action satisfies the pre-condition of the model, does this offer improvements over naively planning with the model?"}}, {"review_id": "ZzbhY2iP2_q", "reviewer": "Reviewer_iJpM", "summary": "The paper presents an approach to reduce planning time by trading off between planning time and model fidelity to solve manipulation tasks. It’s assumed a collection of model-based predictors of varying fidelity are available (e.g. analytic model, learned model, physics-based simulator). Each predictor takes as input a state, action, and parameter vector (for a parametric skill) and predicts the next state that will result. The approach learns models (called model deviation estimators (MDEs)) that predict how much each predictor model will deviate from actual execution. The models are trained based on data collected from executing pre-defined skills in the environment (pick, lift-and-drop, open-drawer). A modified A-star algorithm is defined that incorporates the learned model deviation as a cost so that the search can favor using the least fidelity model it can get away with (i.e. it only needs to use the time-expensive high-fidelity simulator to model more complex tasks like opening a drawer and can use an analytic model for simpler motions pick-and-place of an object). Experiments are performed on a real robot for the tasks of picking and placing a rod into a box, and opening a drawer to drop the rod in the drawer.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n+ The high-level idea of trading off between different models for planning and using simpler models when possible has high utility, enabling redirection of compute power to where it is actually needed (e.g. modeling complex object interactions).\n+ The description of the problem setup and method were clear and easy to follow.\n\nWeaknesses:\n- The models predict deviations based on regression to point-based data, and predict point estimates. This means if a particular skill produces a wide distribution of outcomes given the same inputs, the deviation predictors will average over the outcomes and not predict a useful deviation. The skill of dropping an object can have large variance, depending on the height of the drop and the state of the environment below the object where it’s being dropped. This can be very sensitive to initial conditions, so the object might bounce drastically in different directions even though the same action and environment configuration are repeated several times. The deviation estimator models will only be effective when there is a tight distribution of resulting states on executing a skill.\n- Leaving out the drawer from the analytic model entirely is a strawman comparison. The motivation is to necessitate using a higher fidelity model than the analytic one, but if the analytic model is in fact insufficient to generate a plan, that should happen anyways. Presumably the simulated drawer will be modeled with friction and other dynamics parameters that the analytic kinematic model would ignore, so it could still offer benefits over the analytic. But I think a kinematic model of the drawer should be included also in the analytic model, since the drawer (and its parts) are part of the state/goal space. If the analytic model gave a similar (or better) success rate for planning, it obviates the need for the higher fidelity models, and a more compelling experimental task should be used that necessitates high-fidelity simulation. In short, it’d be more compelling if Figure 4 wasn’t missing any bars.\n- An important baseline is missing: randomly select at certain intervals whether to use the analytic model or the simulator model, and compare that performance to the MDE approach. It might be that a random mixing of analytic and simulator models is just as effective as the more complicated learned MDE approach.\n- It would be more insightful to know at which points in each task the analytic or simulator models are invoked when running the MDE approach. This could demonstrate, for example, that point-to-point motions are easily modeled by an analytic planner while contact-rich manipulations require a more complicated simulator, which would coincide with our intuitions.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The concept of mixing models of different fidelity is a useful idea. However, the formulation of the deviation estimators seems susceptible to failure when skill outcomes have high variance due to environment conditions that are not controllable by the robot (e.g. object dispersion on dropping an object). Further experiments would be required to demonstrate this is not an issue. Further, a key experiment is missing, which is comparison to simpler mixing strategies of using an analytic model and a more complex simulation model, such as simply randomly selecting between the two. It may be that naively switching between analytic and simulation models is similarly effective, and additional experiments are necessary. More insightful analysis can be done regarding where in the task execution different kinds of models are invoked.\n\nUPDATE: In view of the authors' additional experiments, I have updated my scores accordingly. The additional experiments with the analytical model for the drawer and the comparison to a random selection baseline address my main concerns with the experiments. There have been significant revisions in the writing of the paper as well that I feel better communicate the approach and contributions.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "Very good: Comprehensive knowledge of the area"}, "extra_text": {"issues": "Do the timing results in Table 2 include plan attempts that completely timed out at the 5 minute limit? If only one run resulted in the simulator plan timing out, it seems that could drastically skew the results to make both the simulator and MDE planners have higher planning times. For the MDE planner, it would be helpful to include counts of many times it used the analytic planner and how many times it used the simulation planner for each plan produced by the MDE planner.\n\nSimply finding a plan in a certain amount of time is not always the only consideration, the optimality of the plan in terms of shortest path or time optimality may also be a factor. Is there a significant difference between the plans resulting from the analytic/MDE/simulator? To demonstrate this, it would require executing generating multiple plans from each and analyzing the path and execution of each.\n\nThe axes in Fig. 4 should be normalized to be the same (in range [0,1]), currently the right plot axis does not go to 1 which can result in a confused interpretation.\n\nWhy are the property values so different between Table 3 and 4 in appendix? It’s the same robot being modeled in both tasks, and the tasks are not drastically different (both rigid body manipulations), but the joint stiffness/damping values are very different and they use different physics backends.\n"}}, {"review_id": "Vw5BzGj7e7P", "reviewer": "Reviewer_wM8w", "summary": "This paper presents an algorithm for learning a precondition for several transition models with different accuracies and speeds. The main intuition is that because these transition models must trade-off their speed with accuracy, if we can determine the model that would give us a small enough error, then we can choose the one that has the fastest speed. The precondition for model M is defined as the threshold testing on the distance between the true $s_{t+1}$ and the one predicted by M; if this deviation is small enough, the state and action parameter pair is in the precondition of M. The distance function is trained from planning experience using regression, and experiments demonstrate that by using this approach, we can reduce the planning time and achieve higher success rate than the single model approach that uses an analytical or high-accuracy (therefore slow) model alone.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I very much liked the intuition and approach that this paper takes. In manipulation planning, each skill needs to be simulated, and the model must trade-off accuracy with speed. This paper proposes to *learn* how we can make such a trade-off, which I found to be very interesting and novel. However, its weaknesses are the unclear articulation of assumptions and details.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "\nWhile I liked the overall goal and the direction of the paper, I have found several places where this paper should improve to be published.\n\n- It needs to precisely articulate when this approach can be used. It seems that this approach cannot be really used with trajectory optimization algorithms, because they must commit to it. Is it applicable, then, to all tree-search methods? Or is it applicable only to the planning algorithm shown in 4.3?  \n- It is missing the details relevant for learning, especially the feature representation. It does mention using relational features, but looking at the experimental results, I do not fully grasp why relational features, which I presume to be first-order logic-based, are necessary. Also, is the representation tailored only for the given skills and tasks? Or can we apply it in the general case?\n- It seems that the algorithm would heavily depend on the actual distance function used. For example, if your state involves angles, then Euclidean distance would not suffice. What exactly are your state representations in your tasks, and how do you suggest that the users define their distance functions?\n- I found it to be extremely inefficient that it has to solve the same planning problem many times with different models to obtain the training data. Why can't it generalize from other problems? How different are problems in terms of state and goal representations?\n- When the model is inaccurate, one approach you can use is model predictive control. How does this algorithm compare with MPC-like algorithms?\n\n", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Fair: Major revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "issues": "Please see the summary of recommendation.", "reviewer_expertise": "Very good: Comprehensive knowledge of the area"}, "extra_text": {}}, {"review_id": "9nsHp1dV1c", "reviewer": "Reviewer_SzMb", "summary": "This work proposes a method for learning model deviations to define preconditions on the predictive models and thus inform the use of these models during task and motion planning. The problem is broken into two steps: training an \"model deviation estimator\" (MDE), which estimates model deviation for a given state, action, and model and then using the MDE to drive the TAMP search towards states which can be expanded using faster but less accurate models when possible. The paper evaluates the proposed framework in the context of 2 real-world manipulation tasks. ", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The problem statement is well motivated and interesting. The paper is mostly clearly written (though there is a list of recommended fixed included below). The approach taken is interesting, i.e., by using MDE regression instead of classification, and by informing model preconditions rather than skill preconditions. A brief comparison to [1] would be worth adding, however. The experimental results look promising and motivate further exploration of the approach. \n\nThere are some concerns about the experimental setup, however. The lack of an acceptable analytical model for the OpenDrawer task makes for an unsatisfying comparison. Furthermore, by showcasing skills which are so obviously contrasting in predictability (Pick vs LiftAndDrop) it is unclear whether the approach would offer as much time savings given a more diverse set of action primitives and models. In general, the current experimental setup has few skills and considers very few models. \n\nThe idea of using a Multi-Heuristic A* in the context of model selection has been explored in some of the related work, e.g., reference 14 in the paper. Clearer statements about what is different in this work relative to the prior efforts that have followed a similar strategy would strengthen the contribution. \n\nUnderstandably, data collection for such an approach is likely a bottleneck and the authors acknowledge and use some basic data augmentation tool to improve the situation. Given the lack of sufficient experimentation, a discussion on how to more efficiently gather data for more complex setups for this approach would be helpful.\n\nProofreading:\nLine 125: awkwards sentence doesn't make sense; should it be \"... by computing distance ... from predicted state ... and actual state ... using d\"?\nEquation 1: unmatched parathesis\nSection 4.2: Mention which regression models were used for training MDEs or reference section 5.\nLine 174: Shouldn't it be highest value of j?\nLine 254: Model acronyms used here before they are introduced in the next section.\nFigure 4: What are the threshold units in the graph? I assume they are meters but it would be cleaner if written in cm given the text?\nFigure 5: Caption too close to text.\n\n[1] Kim, B., Wang, Z., Kaelbling, L. P., & Lozano-Pérez, T. (2019). Learning to guide task and motion planning using score-space representation. The International Journal of Robotics Research, 38(7), 793-812.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Overall the work proposes an interesting iteration over previous techniques for speeding up TAMP using estimation of model uncertainty but could be stronger in evaluation and better clarify the contribution to some of the related work. ", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "Good: General knowledge of the area"}, "extra_text": {"issues": "A discussion on how to more efficiently gather data for more complex setups for this approach would be helpful. Better support why the current limited experimental setup (in terms of considered skills and models) is sufficient to argue the benefits of the framework."}}]

Author rebuttal:

Meta-review: This paper presents a method for speeding up planning by switching between models at different levels of accuracy.

Quality

(-) Reviewer MmFq and iJpM suggested additional experiments to better evaluate the performance of the proposed method

(+) Experiment with real robots

Clarity:

(-) Reviewers wM8w, SzMb and MmFq suggested many points that should be clarified

Originality:

(+) The proposed approach is novel

Significance:

(+) Several reviews agree that the topic is relevant and impactful

Additional comments:
- I am not sure I understand why do you need the \theta in the formulation of the forward dynamics (line 95)? in an MDP the distribution over the next state is uniquely determined by the current state and action.

---

After rebuttal:
The clarity of the paper was improved and additional experiments were included.
However, Reviewer MmFq is still not fully convinced by the rebuttal.
Please continue to address all remaining feedback.
Decision: Accept (Poster)