Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The authors propose Motion Policy Networks, a learned policy trained on data from classical motion planning solutions. The policy takes as input a configuration space state and a point cloud and outputs configuration space actions. The authors propose additional losses based on motion planning and collect data from motion planning solutions. M\piNets then train a policy and roll out the actions for solve planning problems. This approach is ablated over each loss, data source, and compared to several motion planning techniques and shown to scale well with data and achieve high performance.
strengths_and_weaknesses: The paper is clear and the experiments are well done. The point cloud representation is nice and general, especially compared to representations used in many motion planning and learning + motion planning algorithms. The experimental environments are nice and reasonably complex. The figures are clear as well. I think Figure 3 is informative and clear, that policies have more performance as data increases, whereas structured approaches like MP-Nets outperform in a limited data regime. The ablations are well done and thorough (ablating the loss, data sources, observation, etc) – and informative.

The primary weakness is the framing of the approach. This approach at its core is behavioral cloning in configuration space with additional loses and data training sources. But the authors discuss in detail the motion planning literation as well as learning for motion planning, and ablate and compare to baselines in those fields thoroughly. E.g., they compare to MP-net, which they rightly note requires collision checking, it also has guarantees of falling back on motion planning. 

Instead the authors should be comparing to baselines for learned robotic policies and should be framing the paper accordingly. In framing this way, the authors can discuss data sources more clearly. Questions such as BC data if the data is smooth, noisy, has global coverage are interesting and useful for learning policies. The authors can also show how task-specific losses for motion planning help and hurt. 

Minor comments:
- Can you clarify again why the global planner fails on hybrid problems, when I thought hybrid problems require a global solution to then be smoothed.
- The authors should show an image of the planning problems in the main body of the paper.


robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The authors have written a thorough and clear paper, but it is framed around motion planning when the algorithm is an imitation learned policy with ablations over data sources and task-specific losses. The experiments are thorough and indeed an interesting comparison with motion planning, but it would be useful to better understand how it fits within the imitation learning literature. It would also make comparisons for when learned policies outperform motion planning and when they can learn from motion planning more clear.
issues: See above, primarily with respect to framing.
review_title: null

final_score: rating: Strong Reject: I recommend rejecting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The paper presents an end-to-end deep learning based motion planner that is designed for use with manipulator arms and point cloud representations of obstacles. The goal of the work is to increase planning speed versus previous methods. For example, as compared to sampling based motion planning, graph search, and other forms of optimization. The algorithm is trained using millions of expert trajectories and two expert systems (one based on a sampling based motion planing algorithm, and another based on a hybrid algorithm). Extensive experiments and ablation studies are performed in simulation and there is also a statement that the work has been evaluated on a real robot. 



strengths_and_weaknesses: The main strengths of the paper are as follows:
1) The training set is very large "nearly 300x larger than previous work" according to the paper. 

2) Many different metrics were collected including: success rate, time, rollout target error, collision rat, and smoothness.

3) The number of simulations performed is extensive.

4) The experiments show that using the learned model instead of planning from scratch reduces runtime and increase smoothness (though the model also has decreased performance with respect to other metrics such as success rate). 


The main weaknesses of the paper are as follows:
1) There is little discussion of the relationship between the training and the test sets. Therefore, it is impossible for this reviewer to evaluate if the test set was chosen fairly, and hence, the quality or generality of the experimental results cannot be assessed. 

2) There does not appear to be experimental data/results regarding the performance of the algorithm in dynamic scenarios, despite this being cited as a benefit of the proposed method in the paper, nor when a solution is asked to solve a problem in which a solution does not exist. Testing in cases that do not have solutions is as important as testing in cases that do have solutions.

3) The discussion of graph search and sampling based motion planning algorithms contains many statements that are not true (see A-C below).

4) Despite claims of sim-to-real transfer, the hardware experiments are anecdotal and not empirical. 

5) The  quality or generality of the method with respect to training set size and scope should be discussed in the limitations sections. For example, how close does the training and test set need to be to get decent performance in practice. How  well does training on simulation generalize to testing on hardware?


-- incorrect claims about sampling based motion planning and graph search --

A) The paper states, "Global planners such as RRT [1] optimize for speed and completeness but fall short on optimality." This quote communicates a fundamental misunderstanding about the goals of feasible sampling based motion planning. RRT and other *feasible* sampling based motion planning algorithms are designed to answer the question: "is it possible to reach the goal from the start?" Answering this question in the affirmative is possible given any valid path, and so all valid paths are considered equally "good" by RRT.  So, saying feasible algorithms  "fall short" with respect to optimal is incorrect.  Using RRT when one cares about optimality is simply using the wrong tool for the job.

B) The paper states, "Search-based planning algorithms ... are slow but find least-cost paths... Sampling based planners are typically much faster..." Whether either class of algorithms is "fast" or "slow" depends on how they are used. One can trade speed for better resolution. Moreover, graph search on n nodes is typically many orders of magnitude faster than using sampling based motion planning to create a graph of n nodes from scratch. So, if we fix a lattice graph of a certain resolution then graph-search will require a runtime that scales to the power of the number of C-Space dimensions; on the other hand, if we fix the number of nodes and neighbors then the runtime will remain constant vs. the number of dimensions but the resolution (of nodes per unit hyper-volume) will decrease. 

C) The paper states, "The Global Planner {AIT*} is theoretically complete and optimal, but in practice, we observed several common failures types" and "Global Planner is theoretically complete." This is untrue. AIT* is neither complete nor optimal.  Rather, AIT*, as with all almost-surely asymptotically optimal sampling-based planing algorithms, is *probabilistically complete* and *asymptotically optimal*. Inherent in these definitions are the facts that more planning time will yield better results, on average, and that there is a stochastic nature to the algorithms' performance. It may also be the case that no valid solution exits (in which case the algorithm will not return a solution). 


robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: The idea is interesting and within the scope of the conference, yet there are also a number of errors in the paper regarding previous work, missing details regarding the training and testing process, and claims that are not backed up by experimental results. The main contribution of this paper is the proof of concept demonstration in simulation that end-to-end manipulator planning is possible given point cloud data. While many experiments are run, it is hard to assess the results because there is no indication of how closely the test set resembles the training set. There do not appear to be any empirical results from experiments run in dynamic environments, nor in cases where solutions do not exist, nor on robotic hardware. While the core idea is interesting and a good goal, this version of the paper leaves too much room for improvement.  



issues: 1) Right now the lack of information regarding how training and test sets related makes it hard to understand how to apply this method because there is not evidence of how much a trained model is able to generalize about what it has learned. For example, does training on a table top produce a model that can be used to plan on a counter or on the floor? How close in shape do the objects in the test cases need to be to the objects used in the training examples? Adding such discussion would improve the paper.  Even better would be the addition of empirical assessment of these things.  

2) Deployment in dynamic environments is cited as a motivation for the work multiple times. However, the paper does not appear to contain the results of experiments from dynamic environments. Performing such experiments and reporting how performance changes (e.g., success rate) as a function of, e.g., the speed of environmental changes would improve the paper. Otherwise, claims of deployment in in a dynamic environment should be removed from the paper. 

3) Considering cases where solutions do not exist is just as important as considering cases where solutions do exist. Testing what happens when no solution exists will improve the paper (how often is there a false positive path result that leads to a collision?). 

4) The discussion of sampling based motion planning algorithms contains many statements that are not true. See (A-C) in the "Strengths and weaknesses" section.  This language should be changed to include more nuanced discussion of A-C.

5) The paper states that "MπNets is only trained for single-step prediction, but we use it for closed-loop rollout." Yet, the paper also states that Geometric Fabrics [9] and STORM [10] that "deploy reactive local policies and assume that local decisions will lead to globally acceptable paths" but have worse performance in practice. All three (MπNets, Geometric Fabrics, and STORM) appear to be making local decisions (or single-step prediction) and have an assumption that stitching them into global paths will work. The paper can be improved by adding additional information/discussion about why performance differs between these methods

6) Despite claims of sim-to-real transfer, the hardware experiments are anecdotal and not empirical. Adding empirical hardware experiments would improve the paper by quantitatively demonstrating the sim-to-real transfer. This would also help to alleviate worries of training vs. test sets (that is, if the training set is from simulation and the test set is from real hardware).

7) The  quality or generality of the method with respect to training set size and scope should be discussed in the limitations sections. For example, how close does the training and test set need to be to get decent performance in practice. How  well does training on simulation data generalize to testing on hardware?
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: This paper presents an end-to-end neural motion policy network that learns to generate collision-free trajectories from a point cloud of the scene, the goal state and current joint configuration. It is trained on a large-scale dataset of millions of sample motion planning problems from hundreds of thousands environments. The approach, referred to as M$\pi$Nets, is directly comparable to the MPNets [11] algorithm, showcasing improved performance when sufficient data is available without requiring collision-checking and being able to react in dynamic environments. The data used to train an M$\pi$Net is generated from an expert hybrid planner that leverages the strengths of global and local planners. 
strengths_and_weaknesses: **Strengths**

- The paper is well-written and enjoyable to read. 
- It proposes an interesting solution to difficult motion planning.
- The fact that it is trained on raw point cloud observations makes the approach amenable to sim2real without any additional modifications.
- The evaluation of accuracy and performance of M$\pi$Nets vs. baselines and other SOTA is sufficient and rigorous (except for computation times – discusses in weaknesses)
- Simulated experiments are impressive.
- Inclusion of failures in the video is appreciated. 

**Weaknesses**
- While the comparisons and metrics used to evaluate M$\pi$Nets do a very good job at analyzing its performance based on task/collision-avoidance success, there is no mention about computation time either at training or test time. Only until we go to the appendix we see that M$\pi$Net takes 1 weeks to train with 8 NVIDIA Tesla GPUs. This seems like a very important detail to leave out from the main paper. Furthermore, I would have expected to see a comparison of training/testing time vs. MPNets. How much compute is needed to run the network at test time? Do you need the 8 NVIDIA Tesla GPUs? When proposing an algorithm that requires very heavy computation, one must consider that this might not be accessible or even feasible for certain scenarios or circumstances – hence, the hardware requirements must be transparent.
- Throughout the text it is noted that low performance due to distribution shift could be alleviated by combining the global and local planners. It would be great to see in the paper an explanation as to why this is difficult. Couldn’t you combine the successful demonstrations from both the global and local experts? I assume this might introduce inconsistencies in the data and instead of improving the overall performance it would deteriorate it. Is this the case? It would be great to include such comparison or some extended analysis on this and some suggestions as to how these experts could be combined. Have the authors thought of employing an expert combination strategy like Bayesian Model Combination as done here for an RL setting: https://dl.acm.org/doi/10.5555/3327546.3327623 
- In Table 2, the success rate for M$\pi$Nets tested on the solvable problems seems extremely low 5.5%, could this result be elaborated? What is so fundamentally different between the Hybrid Expert datasets and the MPNets-Styles?
- The two control methods should be mentioned in the main text (frequency for closed-loop control) and further details remaining in appendix.  
- It seems that the points clouds are always uniformly sampled both at training and test time? Is this correct? What is the performance with truly noisy input point clouds?
- The catastrophic and erratic behaviors that are due to distribution shift make it difficult to see this learned end-to-end network working on real-world scenarios. Apart from using/generating more data or allowing corrections to the learned model like DAGGER, there must be a fundamental safety component that allows the robot to at minimum not collide with the environment. Have the authors thought about introducing such a mechanism to the framework?  It would be interesting to see this discussion. 


**Further Improvements/Suggestions**

- To learn a robust policy that is “consistent” with the expert demonstrations the authors might want to consider augmenting the imitation loss with some notions of stability as in:
https://arxiv.org/pdf/2205.14812.pdf
- It would be interesting to include more metrics to evaluate the distinct behaviors of each motion planner, like: obstacle clearance, near-obstacle velocity and motion legibility. 


robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This paper presents an interesting solution to a classical problem in robots - efficient collision free motion planning. This is done by an end–to-end motion policy network that takes as input a point cloud of the scene, the target and the joint configuration of the robot and is learned by a large scale dataset of expert demonstrations provided by planners. While the idea of learning from expert planners is not novel, the performance and range of environments that this motion policy network can handle is unprecedented. The paper needs to be more forthcoming about the computation power necessary to train and run such networks. 

issues: Comments listed in the weaknesses section should be addressed.
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper proposes a neural motion planning learner, which takes a point cloud and a robot configuration as input and outputs the next configuration. The learned model can handle unknown, partially observable, and dynamic environments. Its outperforming performance over various baselines empirically proves its applicability to real-time practical scenarios.
strengths_and_weaknesses: Strengths:
- The performance of the proposed framework is impressive, especially in dealing with dynamic environments.
- Evaluation is exhaustive as various metrics are used, and multiple baselines are compared.

Weaknesses (details are included in the Summary Of Recommendation and Issues sections):
- Relatively simple problems are evaluated. Problems having narrow passages, the most notorious challenge in motion planning, are not sufficiently assessed, in which a learning-based method is expected to work poorly.
- Some points (e.g. task space and the notion of consistency) made in the paper are vague, thus arguable.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The paper is well-organized, and its presentation is clear. Exhaustive evaluations are impressive, and the video demonstration clearly conveys the method's performance. I think the paper can further be improved by addressing the weakness points made above. In particular, whether to be able to handle more intricate problems having narrow passages is of significant concern.
issues: Major comments:
- Harder problems (narrow passages): As pointed out above, most of the problems shown in the paper and the video look relatively simple (wide passages in the configuration space). The only hard one is the limitation scenario in the video. Since the proposed framework is learning-based, its fast computation is not surprising compared to classical planning algorithms. I think a more rigorous analysis of whether challenging problems (e.g. arm going through multiple objects in between) can or cannot be solved is needed to understand how much completeness is sacrificed.
- Consistency (and task space): The authors emphasize the importance of the notion of consistency in many places, but its definition is vague and rather handwavy. It is defined in the paper as: "consistency describes how the motion would change in response to a perturbation in any dimension of the planning problem." Is it a subjective definition? Otherwise, please redefine this notion precisely. This confusion incurs sequential confusion as follows. I couldn't understand why learning in the task space is more beneficial than learning in the configuration space, which has some connection to consistency. Also, the motivation of hybrid planners is unclear. Since this is the primary motivation of the proposed framework, clarifying this point is critical.
- Being able to handle partial observability: The model is trained with fully-observable data, but it is unclear how the ability to handle partial observability at test time emerges.
- Hybrid planner: Because of the confusion about consistency, the fundamental difference between global planner and hybrid planner is vague, making the rest of the paper hard to understand. Moreover, in Section 5, what would be the fundamental differences between problems solvable by either one? Please clarify it. The word "hybrid" comes from the fact that it is a combination of the global planner and the local planner?


Minor comments:
- In the contribution bullet points, "46% better": The word "better" is not well-defined. Is it referring to efficiency or the success rate? Please clarify it.
- In the introduction, "enforce strong assumptions about ...": Please briefly explain the assumptions.
- In Section 2, global planning: The authors state that "optimal paths in configuration space can lead to unintuitive paths in a task space", which may be related to the notion of consistency. What would be intuitive paths in a task space? Distinguishing intuitive paths from unintuitive ones seems arbitrary and subjective. Please provide one example.
- In Section 3.3, "task spaces on the surface of the robot": I couldn't understand the relationship between the task space and the surface of the robot.
- In Section 3.4: Do the closed-loop rollouts mean that the loss is computed from not a single configuration but an entire trajectory?
- In Section 3.4, "We also sample novel scene point clouds on-the-fly during training": I couldn't understand this sentence. Does this mean one problem could have entirely different point clouds from the current scene? I am not sure how learning is possible from it.
- In Section 4.2, global planner: The authors state that the global planner fails if the smoother produces collisions. I am questionable if it is a major and fair cause of failure because it is just a wrong hyperparameter tuning.
- In Section 4.2, hybrid planner: Please clarify "create a consistent configuration space velocity profile", which is hard to understand.
- In Table 3, I couldn't understand why there is such a large gap between hybrid expert and global expert on the environment collision (i.e. 0.94 vs. 13.78), which is related to one of the above comments. Is it still happening even after a careful hyperparameter tuning of the global planner?
- In Section 5.1, the results of the hybrid planner: There are 11%, 2.06%, and 7.94% written in the text, but they don't correspond to any numbers in Table 3. Are they separate results?
- In Section 5.3: 16.41% depends on how the authors considered unknown regions of problems in training. Please explain the method briefly.
- Typo in Section 7: From "end-to-end neural molicy policies", remove "molicy".
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: This paper primarily introduces a novel neural policy architecture - named M$\pi$nets - that can be used to generate robotic motions that achieve a particular goal configuration from a single depth image. To complement this, the authors create a novel extremely-large dataset of motion-planning problems and solutions on which they train their approach. Across extensive experiments, they show that M$\pi$nets not only outperform existing approaches in simulation, but also transfer well to novel tasks with noisy point clouds on real robotic hardware.
strengths_and_weaknesses: Strengths:
- Simplicity of the method: The presented M$\pi$nets architecture and loss functions used for training are rather simple, especially compared to certain baselines, yet perform remarkably well. This simplicity makes the method easy-to-understand and also provides good intuition for why it works.
- Extensive experimental evaluation: The authors' experimental evaluation is extremely thorough and extensive: there is not a single additional result I can think of that I would like to see.
- Well-written paper: The paper's writing and figures are extremely clear and easy-to-follow. Additionally, most important details are provided: I feel that a reader could re-implement the method after carefully studying the paper.

Weaknesses:
No major weaknesses I could think of.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: Overall, I believe this is a strong paper that not only makes several useful contributions to the robotics community, but is also well-presented and easy to follow. In particular, the authors' method and novel dataset offer a new tool towards deploying motion-planning in real-world situations, which has been a goal of the robotics community for several decades. The experimental evaluation is thorough and convincing. I don't see any major flaws that could cause me to recommend rejection.
issues: **Questions**
(Note that these are not issues I have with the paper, but more of a curiosity/question. If possible, it would be great if you could include additional details answering this in either the main paper or supplementary materials.)
- Encoding the target in the point cloud: Is there some reason/intuition for why you decided to encode the target gripper position as part of the point cloud? It seems more straightforward to simply specify the target as a configuration ($q$) vector. Additionally, one of your ablations seems to show that if you remove $q_{t}^{||\cdot||}$ and instead supply this as part of the point cloud, performance degrades substantially. Wouldn't this suggest you can improve performance by also providing the target as an explicit vector instead of as part of the point cloud?
review_title: null

Meta-review: metareview: Summary. 
Motion Policy Networks, a learned policy trained on data from classical motion planning solutions, are proposed by the authors. The authors propose additional losses and collect motion planning data. MpiNets implements a policy to solve planning problems. This method scales well with data and performs well when compared to other motion planning strategies. 
The research describes a deep learning-based motion planner for manipulator arms and obstacle point clouds. The project aims to speed up planning. Comparing sampling-based motion planning, graph search, and other optimizations. Training the algorithm uses millions of expert trajectories and two expert systems (one based on a sampling based motion planing algorithm, and another based on a hybrid algorithm). Extensive simulation experiments and ablation research have been evaluated on a real robot. 
This research provides an end-to-end neural motion policy network that generates collision-free trajectories using a scene point cloud, objective state, and joint configuration. It's trained on millions of motion planning issues from hundreds of thousands of situations. The MNets technique is comparable to the MPNets [11] algorithm, displaying greater efficiency without collision-checking and the capacity to react in dynamic settings 


Strengths 
1. Well-written study with well-executed experiments. 
2. The point cloud representation is excellent and broad compared to various motion planning and learning+motion planning techniques. Attractive and challenging experimental situations. 
3. The training set is "300x greater than previous efforts" 
4. Success rate, time, rollout target error, collision rate, and smoothness were measured. 
5. Experiments demonstrate that utilizing the learnt model decreases runtime and increases smoothness. 
6. It proposes a clever motion planning solution. 
7.  Because it's trained on raw point cloud data, sim2real doesn't need any adjustments. 
8. MNets' accuracy and performance are evaluated against baselines and other SOTA. 
9. The suggested framework excels in dynamic contexts; 
10. The given Mnets design and training loss functions are simple, yet work well. This simplifies the process and explains why it works. 

Weaknesses

1. The training-test set connection is unclear and it cannot be ascertained if the test set was chosen fairly, thus the quality or generalizability of the experimental results cannot be easily evaluated. 
2. There are no experimental data/results on the algorithm's performance in dynamic circumstances, despite this being a benefit of the proposed method. 
3. Graph search and sampling-based motion planning methods contain many erroneous statements as the reviewers have noted.  
4. Hardware trials are anecdotal, notwithstanding claims of simulation-to-real transfer. 
5. The restrictions section should address the method's size and scope. How close must training and test sets be for optimum practice? How well does hardware simulation training generalize? 
6. While comparisons and metrics used to evaluate MNets analyze their performance based on task/collision-avoidance success, computation time during training or testing is not mentioned. 
8. Combining global and local planners can reduce poor performance owing to distribution shift. The report should explain why this is hard. 
9. The two control approaches (frequency for closed-loop control) should be included in the main text. 
10. The authors emphasize consistency, yet its definition is imprecise. Consistency is "how motion changes in response to a planning problem disturbance. This was not shown and needs to be clarified.  
11. The model was trained on fully observable data, thus it's unclear how test-time partial observability occurs. 
12. Due to consistency uncertainty, the difference between global planner and hybrid planner is unclear, making the document have some vague representations.




confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)