From: robot-learning.org/CoRL/2021/Conference/Program_Chairs
title: Paper Decision
decision: Accept (Poster)
comment: This paper presents a method for speeding up planning by switching between models at different levels of accuracy.

Quality

(-) Reviewer MmFq and iJpM suggested additional experiments to better evaluate the performance of the proposed method

(+) Experiment with real robots

Clarity:

(-) Reviewers wM8w, SzMb and MmFq suggested many points that should be clarified

Originality:

(+) The proposed approach is novel

Significance:

(+) Several reviews agree that the topic is relevant and impactful

Additional comments:
- I am not sure I understand why do you need the \theta in the formulation of the forward dynamics (line 95)? in an MDP the distribution over the next state is uniquely determined by the current state and action.

---

After rebuttal:
The clarity of the paper was improved and additional experiments were included.
However, Reviewer MmFq is still not fully convinced by the rebuttal.
Please continue to address all remaining feedback.

From: robot-learning.org/CoRL/2021/Conference/Paper283/Reviewer_MmFq
title: Reviewer reply
comment: Thank you for the response. Several of my issues are resolved, but I still believe the experiments are too simplistic (even with the additional low-fidelity model) to demonstrate that the method will be useful for something that isn't already easy to do with existing methods. I will not be changing my rating.

From: robot-learning.org/CoRL/2021/Conference/Paper283/Area_Chair_YQ1P
metareview: This paper presents a method for speeding up planning by switching between models at different levels of accuracy.

Quality

(-) Reviewer MmFq and iJpM suggested additional experiments to better evaluate the performance of the proposed method

(+) Experiment with real robots

Clarity:

(-) Reviewers wM8w, SzMb and MmFq suggested many points that should be clarified

Originality:

(+) The proposed approach is novel

Significance:

(+) Several reviews agree that the topic is relevant and impactful

Additional comments:
- I am not sure I understand why do you need the \theta in the formulation of the forward dynamics (line 95)? in an MDP the distribution over the next state is uniquely determined by the current state and action.

---

After rebuttal:
The clarity of the paper was improved and additional experiments were included.
However, Reviewer MmFq is still not fully convinced by the rebuttal.
Please continue to address all remaining feedback.
recommendation: Accept (Poster)
confidence: 3: The area chair is somewhat confident

From: robot-learning.org/CoRL/2021/Conference/Paper283/Authors
title: Response to Reviewer iJpM (2/2)
comment: Responses to questions and comments (2/2 due to space constraints)
> Do the timing results in Table 2 include plan attempts that completely timed out at the 5 minute limit?... For the MDE planner, it would be helpful to include counts of many times it used the analytic planner and how many times it used the simulation planner for each plan produced by the MDE planner.

None of the planners timed out in the initial submission's results, though if that happens in the experiments in the revision, we will not include it in the average time. 

> Simply finding a plan in a certain amount of time is not always the only consideration, the optimality of the plan in terms of shortest path or time optimality may also be a factor. Is there a significant difference between the plans resulting from the analytic/MDE/simulator? 

Since we are using A* as our base algorithm, the path costs are comparable in our experimental setup because there are no particularly low-cost actions that can only be simulated using a particular model. The only significant difference across baselines and our method is planning speed and execution reliability. 

> The axes in Fig. 4 should be normalized to be the same (in range [0,1])

We will fix this in the revision. 

> Why are the property values so different between Table 3 and 4 in appendix?

We observed that adding an articulated body in IsaacGym using the flex backend caused issues with simulating the robot control when the only difference is the addition of the articulated body. The physx backend requires much higher gains for the robot to move. We learned this after already running many real-world experiments with MDEs trained against the flex simulation for `RodInBox`, but will use physx for the MDEs in the revision if it is accurate enough.

From: robot-learning.org/CoRL/2021/Conference/Paper283/Authors
title: Response to Reviewer iJpM (1/2)
comment: Thank you for your review and feedback. Below we address your questions and comments. (1/2 due to space constraints)

>  This means if a particular skill produces a wide distribution of outcomes given the same inputs, the deviation predictors will average over the outcomes and not predict a useful deviation. The skill of dropping an object can have large variance, depending on the height of the drop and the state of the environment below the object where it’s being dropped. This can be very sensitive to initial conditions, so the object might bounce drastically in different directions even though the same action and environment configuration are repeated several times. The deviation estimator models will only be effective when there is a tight distribution of resulting states on executing a skill.

MDEs can still be effective for skills that have a wide distribution of outcomes for some states. For most $LiftAndDrop$ transitions, for example, the rod was dropped at many locations between the start location and near the box. The deviation estimation is an overestimate rather than an expectation of deviation to avoid taking actions that might lead to high deviation when the deviation distribution in the training data is high variance.  

 > Leaving out the drawer from the analytic model entirely is a strawman comparison. The motivation is to necessitate using a higher fidelity model than the analytic one, but if the analytic model is in fact insufficient to generate a plan, that should happen anyways. Presumably the simulated drawer will be modeled with friction and other dynamics parameters that the analytic kinematic model would ignore, so it could still offer benefits over the analytic. But I think a kinematic model of the drawer should be included also in the analytic model, since the drawer (and its parts) are part of the state/goal space....

We agree with this assessment that the lack of an analytical model for OpenDrawer is unsatisfying, which was also expressed by another reviewer. We added (results will be in the next revision) an additional low-fidelity model that models the interaction between the grippers and the drawer edge. Including drawers with different friction parameters and more interesting dynamics such as jamming is a good opportunity for future application but due to time constraints we will use the same drawer in the experimental setup, but with the additional low-fidelity model. 

 > An important baseline is missing: randomly select at certain intervals whether to use the analytic model or the simulator model, and compare that performance to the MDE approach...

The random selection baseline was used in Ghosh et al. 2021 (https://arxiv.org/abs/2108.01235) and we will use that as an ablation test in the revision.

> It would be more insightful to know at which points in each task the analytic or simulator models are invoked when running the MDE approach...

This would be an illustrative figure to include: showing the fraction of times a particular model is used for a skill in each task. We will include this data in the revision. The most desirable behavior for selecting parameters for `LiftAndDrop` is to avoid states where pivoting occurs, so our method attempts to avoid those, but the one-queue ablation test does sometimes use the simulator.

From: robot-learning.org/CoRL/2021/Conference/Paper283/Authors
title: Response to Reviewer MmFq (2/2)
comment: Continuation of the previous comment due to character limit.

> The results for the drawer experiment showing that the analytical model can be used even when it completely fails to account for the articulated object is very interesting. Could this framework be used if you instead had two low fidelity models with different limitations? This would be really interesting to see.

Thank you for the suggestion. We described an additional experiment we will run this week and show in the revision where we add an additional low fidelity model that models the drawer articulation.

> What happens when you use your framework using only 1 model? ...

Using our framework with one model would be similar to previous work [11,17,18] that estimates regions where a simple model is accurate, which if the one model is inaccurate, is more reliable than planning with that one model without estimates of where
 the model is accurate.

From: robot-learning.org/CoRL/2021/Conference/Paper283/Authors
title: Response to Reviewer MmFq (1/2)
comment: Thank you for your review and feedback. We respond to the questions and comments here: (1/2)

> The insight that model error underestimation may be better than overestimation is quite interesting and the proposed solution is straightforward. However, it is not clear the method which uses this assumption actually improves performance. Experiments are necessary to verify that underestimation is indeed better.

 During preliminary experiments, we observed that actions with poor model fidelity were chosen when they were close to the threshold due to regression error. Although it would be interesting to compare the implication of different loss functions on plan performance downstream, we leave that for future work due to time and space constraints.

> The experimental results would benefit from an ablation. For instance, it is unclear why the second queue, which only expands using the simplest model, is necessary. The first queue already preferentially expands using the simplest model which satisfies the precondition, so does this not provide enough of a bias toward using the simple model?

The intuition behind the second queue is that the base search will still evaluate successors for actions that require the expensive model. The second queue does not compute the successors for the most expensive model, which allows the faster model evaluation rate shown in Table 2 . We will include an ablation to show the time benefits of using our planner over rejection sampling with the model precondition. 

>  The author's planning framework could very easily incorporate using a binary classifier to predict if the action is in the precondition of the model. The authors claim their method of predicting total state deviation should be ‘more transferable across tasks’ (line 28), however this claim is unsubstantiated.

Placing a rod in a large box requires less precision than placing it in a small container.  Choosing to treat model deviation estimation as regression rather than classification is a choice made to anticipate re-using MDEs in tasks with different precision requirements than the ones initially used for training. We will clarify that we did not compare classification and regression for transferring MDEs across tasks in this paper. 

> I do not think the discussion and comparison of the different models (GP vs MLP etc.) used to learn the MDEs to be particularly useful or relevant to the main contributions of the paper.

We will remove that section to leave room for describing the results of the additional ablation studies described in the general comment above. 

>The tasks used for the experiments are very simplistic and very simple models of pick, place, and drawer-slide actions can easily be used to accomplish them. In fact, such tasks have been done with kinematic motion planners for several decades. A task where the robot is manipulating an object that has interesting dynamics (which actually requires a high-fidelity simulator, at least sometimes) would show the benefit of this method much more clearly.

Although there are certainly tasks with more complex dynamics than ours, we chose our tasks to show our overall idea and how to select between different inaccurate models. Most transitions for $LiftAndDrop$ in our work do require the simulator (or some other dynamics model) to model in-hand pivoting. One target domain we plan to explore in future work is deformable object manipulation, where choice of model and simulator can be quite difficult and affect the computational demands of evaluating the dynamics model.   

> Proofreading and presentation, particularly the partial expansion from open 1.

We will ensure the ordering of models is consistent in the paper and videos in the revision. Thank you for pointing out additional areas for fixes, which will improve clarity.  Furthermore, we will include a graphic in the body of the paper describing our planner's use of partial expansions to build intuition. 


>Videos could use some work. First caption of the planning explanation video is very difficult to parse... The video for the first experiment doesn't seem to complete. 

Our revision will include longer (sentence) captions. Indeed, it does appear the last and most important part of that clip (showing the rod rotating) was removed. 

>The video for the second experiment doesn't include baselines

Initially, the baselines that were able to find a plan looked similar to those found by our method; they were just slower to compute. In the experiments performed in the revision, there were differences so we show an example of a plan that failed from one of the baselines. 

> The fact that there is always a high fidelity model to fall back on means... 

The high fidelity model actually is not always available as a fallback option. For example, even some simulated $LiftAndDrop$ actions do not match the real world because properties such as friction and forces between the gripper and rod become significant.

From: robot-learning.org/CoRL/2021/Conference/Paper283/Authors
title: Response to Reviewer SzMb 
comment: Thank you for your review and feedback. We address your individual comments here. 

>  A brief comparison to [1] would be worth adding, however. 
Both our work and others such as Kim et al. 2019 use the high-level idea of using previous experience to guide planners by predicting the downstream consequences of making decisions during planning. Our work is focused more on plan reliability when executed than optimality, but applies the same high-level idea. We will add this comparison to the related work. 

 >There are some concerns about the experimental setup, however. The lack of an acceptable analytical model for the OpenDrawer task makes for an unsatisfying comparison. 

We agree with the assessment that application of MDEs to $OpenDrawer$ would be more satisfying with an analytical model. As a result, we are adding an additional analytical model that uses the geometry of the drawer and grippers to predict the end state of the drawer after executing $OpenDrawer$. 

>Furthermore, by showcasing skills which are so obviously contrasting in predictability (Pick vs LiftAndDrop) it is unclear whether the approach would offer as much time savings given a more diverse set of action primitives and models. In general, the current experimental setup has few skills and considers very few models. 

We initially chose the skills and models to evaluate our approach in a task where an analytical model is sufficient to solve, and one where the expensive model is necessary to complete the task regardless of action choices. We also wanted to include highly predictable skills where faster simpler models would be sufficient as well as skills that are more difficult to predict without a more complex model.  Additional skills we think would be good applications to show the benefit of MDEs are pushing multiple objects or manipulating deformable objects, but leave this for future work. 

>The idea of using a Multi-Heuristic A* in the context of model selection has been explored in some of the related work, e.g., reference 14 in the paper. Clearer statements about what is different in this work relative to the prior efforts that have followed a similar strategy would strengthen the contribution. 

We will clarify the relationship between our work and others in the related work of the revision. The main difference between our work and related work is using the same planning experience to predict model error for and plan with multiple models instead of a single fast analytical model. CMAX [14] uses hyperspheres to states and actions where the model was inaccurate to decide whether to add a model cost. We use a more general model than hyperspheres, and with a different planner that uses multiple queues rather than adding a model cost, since it can be unintuitive to compare what the cost of using a model is with the cost of an action.  In our MDE representation comparison, we found KNN (which is most similar to the hypersphere representation) to have a worse precision-recall curve than the MLP. 

>Proofreading

We will fix the typos and make the requested clarifications. The assumptions from typos based on context are correct. 

>  Given the lack of sufficient experimentation, a discussion on how to more efficiently gather data for more complex setups for this approach would be helpful.

Given the current need for manual resets for data collection in our experimental setup, we will provide suggestions on data collection in the discussions, such as using offline data, high-fidelity simulators. We will also mention future work we are currently working on that uses a more general state and action space to more directly share MDE across skills, even skills with different parameterizations. Lastly, we will discuss the benefits that can be shown with the skills and models we used in the revised experimental evaluation.

From: robot-learning.org/CoRL/2021/Conference/Paper283/Authors
title: Response to Reviewer wM8w
comment: Thank you for your time reading our paper and providing feedback. We address your individual comments here. 

>It needs to precisely articulate when this approach can be used. It seems that this approach cannot be really used with trajectory optimization algorithms, because they must commit to it. Is it applicable, then, to all tree-search methods? Or is it applicable only to the planning algorithm shown in 4.3?

MDEs to describe model preconditions are widely applicable for most model-based planning. They were designed with search-based planning in mind, but model preconditions described by MDEs can be used as constraints in some non-linear optimization algorithms. We previously used MDEs as hard constraints for rejection sampling but observed significant speedup using the planning algorithm in 4.3 by prioritizing inexpensive expansions. We also considered adding a model cost (similar to Vemula et al. 2020) to the action cost, but found it hard to compare distance cost functions and a model cost. We will make the utility of MDEs and of our planning algorithm clear in the discussion of our revision. 

> It is missing the details relevant for learning, especially the feature representation. It does mention using relational features, but looking at the experimental results, I do not fully grasp why relational features, which I presume to be first-order logic-based, are necessary. Also, is the representation tailored only for the given skills and tasks? Or can we apply it in the general case?

We found experimentally better precision-recall curves using the relative features than absolute positions. The representation of the state we use (pairwise distances between objects) is fairly general and we believe applies to many manipulation skills and tasks where the most important information is the relative poses of the objects and the action. 

> What exactly are your state representations in your tasks, and how do you suggest that the users define their distance functions?

The distance function is indeed very important to the utility of MDEs. Describing the distance between states is very task-specific and finding a general one is an open research problem. One way to add angle error is to add the minimum distance between the observed rotation and the predicted rotation to the position error. If the identities of the object are not important (ex. One rod needs to be in the box, but it does not matter which rod) a linear assignment problem can be used to compute the distance, such as in Huang et al. 2019 (https://ieeexplore.ieee.org/abstract/document/8793946) 
> I found it to be extremely inefficient that it has to solve the same planning problem many times with different models to obtain the training data. Why can't it generalize from other problems? How different are problems in terms of state and goal representations?

MDEs are reused across planning problems, which we will clarify in the revision. For example, we share the same MDEs for $Pick$ and for $LiftAndDrop$ across `RodInDrawer` and `RodInBox`. It costs no additional real-world data to train a new model on the existing transitions. Actions that share a parameter space, such as $Pick$ and $LiftAndDrop$ can theoretically share MDEs but we were not able to get a version working and tested in time for the revision. We are currently working on MDEs that share a state and action space, so they can be shared across skills, though those results will not be included in this paper. 

> How does this algorithm compare with MPC-like algorithms?

Our method is compatible with MPC. A method using MPC to address model error would benefit from our work if model error would either cause suboptimal actions, states that are hard or impossible to recover from, or would benefit computationally from a lower replanning frequency.

From: robot-learning.org/CoRL/2021/Conference/Paper283/Authors
title: General Response to Reviewers
comment: We thank the reviewers for their comments and suggestions. Thank you for identifying several areas of improvement that we address generally here, and then expound on in individual comments tailored to each review. We describe the experiments we will run to give reviewers a chance to offer additional feedback before we run them and add the results to the revision in about a week. 

 First, most reviewers expressed concern that the target domains were insufficiently complex to evaluate the algorithm, especially the drawer opening environment, which uses the simulator for $OpenDrawer$ when using our method. We added an additional low-fidelity analytical model that can model $OpenDrawer$ for `RodInDrawer`, which we will add results for to show how MDEs can be used to select between two low-fidelity models and a high-fidelity model. 

Second, we want to clarify that the rod rotates in the gripper for some grasps, requiring a dynamics model for most transitions, and will make the role of MDEs for predicting those interactions more clear in the revised video and paper. 

Lastly, we will run our experiments with additional ablation studies suggested by Reviewer iJpM and Reviewer MmFq to show the benefits of using MDEs, and of using the planning algorithm we propose.

From: robot-learning.org/CoRL/2021/Conference/Paper283/Reviewer_wM8w
summary: This paper presents an algorithm for learning a precondition for several transition models with different accuracies and speeds. The main intuition is that because these transition models must trade-off their speed with accuracy, if we can determine the model that would give us a small enough error, then we can choose the one that has the fastest speed. The precondition for model M is defined as the threshold testing on the distance between the true $s_{t+1}$ and the one predicted by M; if this deviation is small enough, the state and action parameter pair is in the precondition of M. The distance function is trained from planning experience using regression, and experiments demonstrate that by using this approach, we can reduce the planning time and achieve higher success rate than the single model approach that uses an analytical or high-accuracy (therefore slow) model alone.
strengths_and_weaknesses: I very much liked the intuition and approach that this paper takes. In manipulation planning, each skill needs to be simulated, and the model must trade-off accuracy with speed. This paper proposes to *learn* how we can make such a trade-off, which I found to be very interesting and novel. However, its weaknesses are the unclear articulation of assumptions and details.
originality: Good: Some advance.
technical_quality: Fair: Significant additional evidence required to support the claims.
clarity_of_presentation: Fair: Major revision needed to improve organization or clarity.
impact: 4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.
recommendation: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
summary_of_recommendation: 
While I liked the overall goal and the direction of the paper, I have found several places where this paper should improve to be published.

- It needs to precisely articulate when this approach can be used. It seems that this approach cannot be really used with trajectory optimization algorithms, because they must commit to it. Is it applicable, then, to all tree-search methods? Or is it applicable only to the planning algorithm shown in 4.3?  
- It is missing the details relevant for learning, especially the feature representation. It does mention using relational features, but looking at the experimental results, I do not fully grasp why relational features, which I presume to be first-order logic-based, are necessary. Also, is the representation tailored only for the given skills and tasks? Or can we apply it in the general case?
- It seems that the algorithm would heavily depend on the actual distance function used. For example, if your state involves angles, then Euclidean distance would not suffice. What exactly are your state representations in your tasks, and how do you suggest that the users define their distance functions?
- I found it to be extremely inefficient that it has to solve the same planning problem many times with different models to obtain the training data. Why can't it generalize from other problems? How different are problems in terms of state and goal representations?
- When the model is inaccurate, one approach you can use is model predictive control. How does this algorithm compare with MPC-like algorithms?


issues: Please see the summary of recommendation.
reviewer_expertise: Very good: Comprehensive knowledge of the area

From: robot-learning.org/CoRL/2021/Conference/Paper283/Reviewer_SzMb
summary: This work proposes a method for learning model deviations to define preconditions on the predictive models and thus inform the use of these models during task and motion planning. The problem is broken into two steps: training an "model deviation estimator" (MDE), which estimates model deviation for a given state, action, and model and then using the MDE to drive the TAMP search towards states which can be expanded using faster but less accurate models when possible. The paper evaluates the proposed framework in the context of 2 real-world manipulation tasks. 
strengths_and_weaknesses: The problem statement is well motivated and interesting. The paper is mostly clearly written (though there is a list of recommended fixed included below). The approach taken is interesting, i.e., by using MDE regression instead of classification, and by informing model preconditions rather than skill preconditions. A brief comparison to [1] would be worth adding, however. The experimental results look promising and motivate further exploration of the approach. 

There are some concerns about the experimental setup, however. The lack of an acceptable analytical model for the OpenDrawer task makes for an unsatisfying comparison. Furthermore, by showcasing skills which are so obviously contrasting in predictability (Pick vs LiftAndDrop) it is unclear whether the approach would offer as much time savings given a more diverse set of action primitives and models. In general, the current experimental setup has few skills and considers very few models. 

The idea of using a Multi-Heuristic A* in the context of model selection has been explored in some of the related work, e.g., reference 14 in the paper. Clearer statements about what is different in this work relative to the prior efforts that have followed a similar strategy would strengthen the contribution. 

Understandably, data collection for such an approach is likely a bottleneck and the authors acknowledge and use some basic data augmentation tool to improve the situation. Given the lack of sufficient experimentation, a discussion on how to more efficiently gather data for more complex setups for this approach would be helpful.

Proofreading:
Line 125: awkwards sentence doesn't make sense; should it be "... by computing distance ... from predicted state ... and actual state ... using d"?
Equation 1: unmatched parathesis
Section 4.2: Mention which regression models were used for training MDEs or reference section 5.
Line 174: Shouldn't it be highest value of j?
Line 254: Model acronyms used here before they are introduced in the next section.
Figure 4: What are the threshold units in the graph? I assume they are meters but it would be cleaner if written in cm given the text?
Figure 5: Caption too close to text.

[1] Kim, B., Wang, Z., Kaelbling, L. P., & Lozano-Pérez, T. (2019). Learning to guide task and motion planning using score-space representation. The International Journal of Robotics Research, 38(7), 793-812.

originality: Good: Some advance.
technical_quality: Good: Generally convincing, but some additional experiments or analyses needed.
clarity_of_presentation: Good: Minor revision needed to improve organization or clarity.
impact: 3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.
recommendation: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
summary_of_recommendation: Overall the work proposes an interesting iteration over previous techniques for speeding up TAMP using estimation of model uncertainty but could be stronger in evaluation and better clarify the contribution to some of the related work. 
issues: A discussion on how to more efficiently gather data for more complex setups for this approach would be helpful. Better support why the current limited experimental setup (in terms of considered skills and models) is sufficient to argue the benefits of the framework.
reviewer_expertise: Good: General knowledge of the area

From: robot-learning.org/CoRL/2021/Conference/Paper283/Reviewer_MmFq
summary: This paper presents a method for planning robot manipulation tasks using multiple forward models of different fidelities. The authors propose learning to predict the expected total error of a model's prediction for a given state and action. The authors then propose a planning method which simultaneously rejects actions which have high predicted error and biases the search towards using lower fidelity models. The approach is demonstrated on two experiments each with two models, showing improvements in planning speed over using only a high fidelity model, while remaining competitive in the success rate of the generated plans. 
strengths_and_weaknesses: Strengths
-The paper is well motivated, planning with multiple imperfect models is an area of great interest to robotics. 

-The planning method utilizing different models seems quite a sound idea. It is also an interesting idea that we might want to preferentially use different models for different situations. 

-Experiments demonstrate that the method is useful for speeding up planning without compromising performance on a real robot with a very small dataset. 

Weaknesses
-The main weakness of this paper is the clarity of writing and presentation. The paper would benefit from thorough revisions and editing for clarity in all sections. A few examples are described in the “Issues” below.

-The insight that model error underestimation may be better than overestimation is quite interesting and the proposed solution is straightforward. However, it is not clear the method which uses this assumption actually improves performance. Experiments are necessary to verify that underestimation is indeed better.

-The experimental results would benefit from an ablation. For instance, it is unclear why the second queue, which only expands using the simplest model, is necessary. The first queue already preferentially expands using the simplest model which satisfies the precondition, so does this not provide enough of a bias toward using the simple model?

-The author's planning framework could very easily incorporate using a binary classifier to predict if the action is in the precondition of the model. The authors claim their method of predicting total state deviation should be ‘more transferable across tasks’ (line 28), however this claim is unsubstantiated.

-I do not think the discussion and comparison of the different models (GP vs MLP etc.) used to learn the MDEs to be particularly useful or relevant to the main contributions of the paper. 

-The tasks used for the experiments are very simplistic and very simple models of pick, place, and drawer-slide actions can easily be used to accomplish them. In fact, such tasks have been done with kinematic motion planners for several decades. A task where the robot is manipulating an object that has interesting dynamics (which actually requires a high-fidelity simulator, at least sometimes) would show the benefit of this method much more clearly.

originality: Good: Some advance.
technical_quality: Good: Generally convincing, but some additional experiments or analyses needed.
clarity_of_presentation: Poor: Difficult to follow and understand.
impact: 3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.
recommendation: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
summary_of_recommendation: Overall the idea of using multiple models to efficiently find plans for manipulation tasks is interesting and the approach has some technical novelty. However, the tasks used in the experiments are very simple and can easily be done using traditional pick/place/open-drawer actions generated by existing planners with very simple models, thus the usefulness of this method over state-of-the-art is not clear. Also, the paper is very hard to read and the clarity needs to be significantly improved.
issues: -The caption of figure 1 describes a figure on the left which does not appear to exist

-Several points in the paper, the notation switches from M0 being the high fidelity model and MK being the low fidelity model  to the other way around. (For example figure 1 vs line 96 vs line 177). This makes it very difficult in section 4.3 to understand exactly which model is being used when, I am left having to guess the author's intentions. 

-I had a great deal of difficulty understanding section 4.3, particularly the partial expansion from open 1. Some sort of graphical explanation / a figure or algorithm might be very useful for getting the information across. 

-Videos could use some work. First caption of the planning explanation video is very difficult to parse for instance. 

-For the videos of the real robot experiments, the video for the first experiment doesn’t seem to complete, I am unsure what to make of it. The Video for the second experiment doesn’t include baselines. If time is an issue I would recommend changing the video speed for the experiments to be 4x real time (or something). 

-The fact that there is always a high fidelity model to fall back on means that in the worst case it can likely find a good plan albeit slowly. The results for the drawer experiment showing that the analytical model can be used even when it completely fails to account for the articulated object is very interesting. Could this framework be used if you instead had two low fidelity models with different limitations? This would be really interesting to see. 

-What happens when you use your framework using only 1 model? This should only expand nodes for which the action satisfies the pre-condition of the model, does this offer improvements over naively planning with the model?
reviewer_expertise: Very good: Comprehensive knowledge of the area

From: robot-learning.org/CoRL/2021/Conference/Paper283/Reviewer_iJpM
summary: The paper presents an approach to reduce planning time by trading off between planning time and model fidelity to solve manipulation tasks. It’s assumed a collection of model-based predictors of varying fidelity are available (e.g. analytic model, learned model, physics-based simulator). Each predictor takes as input a state, action, and parameter vector (for a parametric skill) and predicts the next state that will result. The approach learns models (called model deviation estimators (MDEs)) that predict how much each predictor model will deviate from actual execution. The models are trained based on data collected from executing pre-defined skills in the environment (pick, lift-and-drop, open-drawer). A modified A-star algorithm is defined that incorporates the learned model deviation as a cost so that the search can favor using the least fidelity model it can get away with (i.e. it only needs to use the time-expensive high-fidelity simulator to model more complex tasks like opening a drawer and can use an analytic model for simpler motions pick-and-place of an object). Experiments are performed on a real robot for the tasks of picking and placing a rod into a box, and opening a drawer to drop the rod in the drawer.
strengths_and_weaknesses: Strengths:
+ The high-level idea of trading off between different models for planning and using simpler models when possible has high utility, enabling redirection of compute power to where it is actually needed (e.g. modeling complex object interactions).
+ The description of the problem setup and method were clear and easy to follow.

Weaknesses:
- The models predict deviations based on regression to point-based data, and predict point estimates. This means if a particular skill produces a wide distribution of outcomes given the same inputs, the deviation predictors will average over the outcomes and not predict a useful deviation. The skill of dropping an object can have large variance, depending on the height of the drop and the state of the environment below the object where it’s being dropped. This can be very sensitive to initial conditions, so the object might bounce drastically in different directions even though the same action and environment configuration are repeated several times. The deviation estimator models will only be effective when there is a tight distribution of resulting states on executing a skill.
- Leaving out the drawer from the analytic model entirely is a strawman comparison. The motivation is to necessitate using a higher fidelity model than the analytic one, but if the analytic model is in fact insufficient to generate a plan, that should happen anyways. Presumably the simulated drawer will be modeled with friction and other dynamics parameters that the analytic kinematic model would ignore, so it could still offer benefits over the analytic. But I think a kinematic model of the drawer should be included also in the analytic model, since the drawer (and its parts) are part of the state/goal space. If the analytic model gave a similar (or better) success rate for planning, it obviates the need for the higher fidelity models, and a more compelling experimental task should be used that necessitates high-fidelity simulation. In short, it’d be more compelling if Figure 4 wasn’t missing any bars.
- An important baseline is missing: randomly select at certain intervals whether to use the analytic model or the simulator model, and compare that performance to the MDE approach. It might be that a random mixing of analytic and simulator models is just as effective as the more complicated learned MDE approach.
- It would be more insightful to know at which points in each task the analytic or simulator models are invoked when running the MDE approach. This could demonstrate, for example, that point-to-point motions are easily modeled by an analytic planner while contact-rich manipulations require a more complicated simulator, which would coincide with our intuitions.


originality: Good: Some advance.
technical_quality: Good: Generally convincing, but some additional experiments or analyses needed.
clarity_of_presentation: Very Good:  Well organized and clear.
impact: 3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.
recommendation: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
summary_of_recommendation: The concept of mixing models of different fidelity is a useful idea. However, the formulation of the deviation estimators seems susceptible to failure when skill outcomes have high variance due to environment conditions that are not controllable by the robot (e.g. object dispersion on dropping an object). Further experiments would be required to demonstrate this is not an issue. Further, a key experiment is missing, which is comparison to simpler mixing strategies of using an analytic model and a more complex simulation model, such as simply randomly selecting between the two. It may be that naively switching between analytic and simulation models is similarly effective, and additional experiments are necessary. More insightful analysis can be done regarding where in the task execution different kinds of models are invoked.

UPDATE: In view of the authors' additional experiments, I have updated my scores accordingly. The additional experiments with the analytical model for the drawer and the comparison to a random selection baseline address my main concerns with the experiments. There have been significant revisions in the writing of the paper as well that I feel better communicate the approach and contributions.
issues: Do the timing results in Table 2 include plan attempts that completely timed out at the 5 minute limit? If only one run resulted in the simulator plan timing out, it seems that could drastically skew the results to make both the simulator and MDE planners have higher planning times. For the MDE planner, it would be helpful to include counts of many times it used the analytic planner and how many times it used the simulation planner for each plan produced by the MDE planner.

Simply finding a plan in a certain amount of time is not always the only consideration, the optimality of the plan in terms of shortest path or time optimality may also be a factor. Is there a significant difference between the plans resulting from the analytic/MDE/simulator? To demonstrate this, it would require executing generating multiple plans from each and analyzing the path and execution of each.

The axes in Fig. 4 should be normalized to be the same (in range [0,1]), currently the right plot axis does not go to 1 which can result in a confused interpretation.

Why are the property values so different between Table 3 and 4 in appendix? It’s the same robot being modeled in both tasks, and the tasks are not drastically different (both rigid body manipulations), but the joint stiffness/damping values are very different and they use different physics backends.

reviewer_expertise: Very good: Comprehensive knowledge of the area