Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The authors propose a new learning from demonstration architecture that creates a structured policy capable of controlling robots with high DoF from a limited number of demonstrations. The paper presents the architecture and learnable parameters of Neural Geometric Fabrics (NGF) by first introducing the relevant prior work on Geometric Fabrics. An extensive comparison of NGF and several baselines is presented demonstrating the success rate and safe deployment of NGF on three tasks.
strengths_and_weaknesses: # Strengths
* The paper is well written and shows a clear progression from the relevant prior work to the proposed architecture. The authors make a good effort to simplify the complexity of the method for the reader.
* The authors select 4 baselines for comparison with NGF and provide a comprehensive evaluation of the results. 
* The supplementary material includes a video verifying the robot deployments and further evaluations of the baselines and NGF method along with architecture parameters and algorithms.

# Weaknesses
There are no large or obvious weaknesses to this paper. However, another baseline for comparison that induces structure for learning motion trajectories would be helpful for the reader to validate the improvements NGFs make to the learning from demonstration area. The only method that comes to mind is "Conditional Neural Motion Primitives" although there are likely better ones. 
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This paper introduces a novel, structured method for learning from demonstration and proves its sample efficiency and ability to control robots with high degrees of freedom. The inclusion of multiple baselines and a thorough experimental section make this a considerable contribution. 
issues: * In Figure 2, the numbering in the caption does not easily follow the graphic. For readability could numbering or steps be added to the graphic.
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes to leverage learning-from-demonstration to learn the parameters of a Geometric-Fabric motion policy, parametrized by neural networks. The main motivation of this work is to leverage the structure provided by geometric fabrics in robot motion generation to address the problem of designing dexterous manipulation skills in high-DoF robotic systems. The method is tested on three different manipulation tasks that involving picking, pushing and extraction skills. 
strengths_and_weaknesses: **Strengths**
1. Real experiments on hardware:
One of the key aspects of this paper is the real experiments reported in Section 6 and shown in the paper. On the one hand, it is very clear how introducing the structure of geometric fabrics brings benefits when learning dexterous manipulation skills, which is very clearly observed in Fig. 3. On the other hand, the information provided in Appendix and the available code may provide the opportunity to reproduce the results reported in the paper and/or employ the proposed method in other works. 

2. Facilitate the design of geometric fabrics via LfD:
It is definitely important to highlight that the design of the parameters of geometric-fabric policies may be tedious, and leveraging LfD to address this problem brings some benefits when using this robot motion generator in practical settings. 


**Weaknesses** 
1. Unclear contributions:
   * The paper clearly states three contributions in lines 59-65, claiming novelty on the use of LfD for geometric-fabrics policies, thorough analysis of the policy structure, and the first framework to train and generate joint-space motions on high-DoF dexterous manipulation using learning methods. Below I explain the reasons why I disagree with some of these claims:
      - Using LfD and Geometric fabrics seems to be relatively new. However, the combination of LfD to learn RMP parameters (a very similar approach, as acknowledged in the paper) was already proposed by Rana et al [11]. There is nothing particularly new in the design of the networks or learning process that is specially designed for Geometric Fabrics. In this sense, it is hard to see this as a novelty. Going beyond this, Geometric fabrics is an approach proposed in [12] and [13], the tree-like structure to combine different policies was proposed in the RMPflow work [45] and also improved in [10]. 

     - Analysis of the importance of the policy structure is somehow confusing. On the one hand, I agree that comparing agains FF and LSTM networks is a good way to show that policies with some meaningful structure provide benefits when it comes to learn motion policies. However, the comparison against RMP, which is also an approach that provides a strong policy structure is not analyzed in detail. It is unclear why such big differences exist given the fact that RMPs may also be learned from demonstrations and have shown successful experiments in past works. The paper should provide clearer evidence or a deep discussion on the possible reasons of the observed performance differences.

2. Related work:

   Although the paper does a good job by reviewing different works using IL and RL to learn dexterous manipulation skills, there are some points I think deserve special attention:

   - Two recent works on learning dexterous manipulation (on high-DoF robots) from demonstration are missing [a,b] (which I think are very close, in practical terms, to the concepts and experiments reported in this paper):
      [a] A. Gupta, C. Eppner, S. Levine and P. Abbeel, "Learning dexterous manipulation for a soft robotic hand from human demonstrations," 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016
      [b] P. Ruppel and J. Zhang, "Learning Object Manipulation with Dexterous Hand-Arm Systems from Human Demonstration," 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020.

   - In the review of IL approach, the paper claims that one of the weaknesses of the related works is the data efficiency, yet the proposed approach in this paper was tested with 80 demonstrations for one of the experiments, which does not seem any different from other IL approaches. 

   - In lines 102-104, the paper claims that NFG provides a more “appropriate inductive bias” than RMPs. But it is unclear what the paper means with “more appropriate”. This claim needs to be clarified. 

   - In more general terms, the paragraph on exploiting structures does not provide any insights on the differences between the structure that NFGs bring when compare to more classical motion primitives such as DMP and ProMPs, which are cited but not discussed. 

3. Experiments:
As pointed out above, the comparison against FF and LSTM networks clearly show the advantages of having structure in the robot policy representation, this is a great result to report. However, I would like to suggest two important improvements in the experiments:
   - Figure 3 and 4 are all about showing the benefits of including structure for task performance and data efficiency. However, if there is an approach in the literature that stands out as a method that is data efficient and includes structure is DMPs, cited in this paper. DMPs provide a second-order DS structure, can be learned from a single demonstration (in some mild cases) and are mainly learned using LfD. So, this seems to be a very good baseline to use to compare against to. Either if the DMP is learned in task or joint space, this may provide some additional insights on the performance of NFG against other approaches that provide strong structure. 
    - It is unclear if RMPs parameters are learned as proposed by Rana et al, or if this paper considers manually-designed RMPs for comparison purposes.
    - Figure 4, in my opinion, shows that NFG are **not** data efficient. They seem to be better than FF and LSTM, but considering success, in terms data efficiency, when using 60 human demonstrations is not data efficient in LfD literature (consider DMP or ProMPs, which can be trained with a handful of demonstrations). More importantly, real applications of LfD approaches that demand the human teacher to provide more than 10-15 demonstrations are not practical. This should be considered a limitation of the approach and discussed in the corresponding section. 

*Lack of details in proposed approach*
- Clarify why the metric-weighted projection matrix $\mathbf{P}_e$ boils down to an identity-matrix metric in line 193. 
- Line 217 refers to the appendix to understand the policy optimization and loss function, but in my opinion, both are core components of the proposed approach and should appear in the main text. 

     
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: The paper leverages LfD to learn the parameters of Geometric-Fabric policies. In this regard, the paper does not clearly state the main differences w.r.t all the previous works on RMPs and Geometric-Fabrics, as some of them (in particular Rana’s work) highly overlap on the ideas and concepts that this paper builds on. Along a similar line, due to the high overlap with previous works, it seems that several claims about novelty are hard to see. Last but not least, the experiments fall short on providing strong evidence on data efficiency, and there is a lack of comparison against more classical movement primitives, that theoretically speaking, may also provide strong structure and data efficiency, as claimed for NFGs.

------

**Post-rebuttal recommendation**

Thank you very much for the rebuttal and interesting discussion. I appreciate very much the time and effort invested during the rebuttal phase. Below you can find my thoughts on my final recommendation:

1. As I stated in the rebuttal, GFs and RMPs are very similar geometric control approaches, with the former being a sort of generalization of the latter. As such, nearly every extension made about RMPs (RMPflow, RMP2, RMP learning via demonstrations) can be applied to GFs. In fact, the paper leverages RMPflow computations and similarly, the proposed extension to learn the GF parameters from demonstrations is very close to the RMP extension of Rana et al.,(2019). In this regard, I find the authors' reply unsatisfying.

2. Another aspect that is not fully addressed is sample efficiency. In my experience, well-known LfD approaches are easily trained with 10-15 demonstrations (e.g., DMPs, GMMs, CNPs, ProMPs, to mention a few) for fairly complicated tasks. Some of these approaches, extended to complex settings like Riemannian manifolds, have been trained with such small datasets. So, in this regard, 60+ demonstrations do not align with sample efficiency claims. Thus, the authors' rebuttal was unsatisfying.

3. I fully agree with the authors' on that GFs can be seen as inductive bias in robot motion skill learning (as discussed in my rebuttal). However, this is not specific to this paper, but it is a contribution of the GFs papers themselves. On a similar line, if such inductive bias were a game changer when learning this kind of policies, the paper falls short on providing evidence on this regard: RMPs, which also bring inductive bias to the learning procedure, seem to perform similarly w.r.t NN or LSTM (in some cases). Then it is unclear why the inductive bias that RMPs bring to the problem do not provide improvements over these baselines and why GFs do instead.

4. Last but not least. I think the work reported in this paper is very interesting, it has good merits, as I pointed out in my review. But I would like to also second the feedback of reviewer 1QRC, which highlights other aspects that I overlooked in the paper and have not been properly addressed.

Then, I keep my recommendation. 
issues: The main issues of this paper are: unclear contributions (it is unclear how much novelty the paper brings when compared to all the RMP and Geometric Fabrics literature), confusing claims on related works and missing references, and the provided experiments. Please see my comments above for suggestions.
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes to learn the parameters of policies encoded via geometric fabrics from demonstrations. Specifically, the paper focuses on learning the policy parameters for a hand-and-arm system, whose motion is generated by two geometric fabrics describing the nominal behavior of the system in task and joint space and a geometric fabric inducing an additional potential function. The difference geometric fabrics are then combined following the principles of RMPflow, where the Riemannian motion policies are replaced by the geometric fabrics. The paper then proposes to encode the geometric fabrics' parameters as neural networks, and to learn them from demonstrations. The paper tests the proposed approach in 3 different manipulation tasks, and assesses its performance with respect to several neural-network-based policies and to a Riemannian motion policy.
strengths_and_weaknesses: # Strengths

- The proposed approach is interesting as it allows the learning of the parameters of a geometric-fabric-based policy from demonstrations. This may be beneficial as geometric fabrics may be cumbersome to design by hand. 
- The proposed approach is nicely illustrated with 3 real-world experiments involving a complex robotic system (multi-fingered hand mounted on a robotic arm). The paper also made an nice effort in comparing their approach with respect to other state-of-the-art policies on the real system. 

# Weaknesses
- The novelty of the paper is rather limited: It mostly follows the ideas presented in [11] (Rana et al, CoRL'20), where the parameters of Riemannian motion policies were encoded as neural networks and learned from demonstrations. As geometric fabrics generalize Riemannian motion policies [12], the contributions of the paper are rather incremental.
- The presented geometric-fabric-based policy is very specific to the system and tasks at hand. Although the presented approach may also be used for geometric-fabric-based policy specified for other systems and tasks, the paper does not mention it. Consequently, it is also unclear how the proposed approach may perform for and scale to different systems/tasks. 
- For example, would the proposed approach scale to tasks considering more than 3 geometric fabrics? This is unclear to me, as this would result to an increase of the number of parameters to learn, which may be unfeasible without a very large number of demonstrations (considering the large number of demonstrations used for the tasks considered in the paper, see next comment).
- 80 demonstrations are used for the two first tasks evaluated in the paper. Learning-from-demonstration (LfD) approaches usually use a handful of trials, and aim at facilitating the learning of skills from few demonstrations (including generalizable skills). In that sense, providing 80 demonstrations to essentially learn a picking and a pushing task seem very cumbersome and not adapted for a viable LfD approach. This is even more striking when considering that only 6 demonstrations were provided for the third task considering in the paper. 
- Following the previous remark, the analysis on sample efficiency conducted in the first task tend to show that a very large number of demonstrations is needed to obtain a decent performance on the first task. Although the proposed approach is the most sample-efficient according to this analysis, the required number of demonstrations remain considerable. In my opinion, this is a clear limitation, especially when considering that the demonstrations were acquired from a human (via teleoperation).
- The paper claims to learn high-dimensional policies. Although the hand-and-arm system originally has 23 DoFs, the dimensionality of the geometric fabric describing the nominal behavior of the system in task space and of the geometric fabric encoding the potential function is reduced to 8 by considering the position of the end-effector (3 dimensions) and the eigenspace of the hand (5 dimension). These two low-dimensional geometric fabrics clearly encode the most important part of the motion, as the remaining geometric fabric serves as a residual policy, filling in the nullspace of the 8-dimensional space. Therefore, I find the claim of learning high-dimensional policies somewhat exaggerated, as a lower-dimensional space built on prior knowledge is considered for the most important part of the motion.
- Additional details should be given in order to make the paper self-contained. For example, as learning the geometric-fabric-based policies from demonstrations is the main contribution of the paper, the considered loss should appear in the main text. Similarly, it would be beneficial to define the imitation error in the main text. The input of all networks are not detailed in the main paper, while features and states are mentioned as inputs in the Appendix. Additional explanations on this would be appreciated. Which features and states are considered here? Also, the roles of $[M_*, \pi_*]$ (section 4.2) and the decomposition of the terms $\pi_k$ (Figure 2) are not explained. 
- The variable $\pi$ is used to describe the geometries (Section 3, 4) and the policy (Section 5), which may be confusing for the reader.
- In Section 2, in the "exploiting structure" paragraph: probabilistic movement primitives ([33], [34]) are not dynamical-system-based approaches. In general, the paper tends to cite long lists of papers without explaining each of them in details (e.g., [1-9], [29-39]). Selectively citing and explaining some of these papers may be more interesting when describing the related work.
- As the paper is concerned with learning from demonstrations and does not consider reinforcement learning, the background section on reinforcement learning does not seem necessary.

## Open questions
- How many parameters are learned for the final policy?
- The low performance of Riemannian motion policies compared to geometric fabrics (and other approaches) in the presented experiments was surprising to me, as geometric fabrics are essentially an improvement on Riemannian motion policies. Is there an explanation for this?

## Cosmetics
- Consider displaying the y-axis from 0 to 100% for all success rate graphs.
- The labels in the figures are generally too small.
- Please improve the format of the references. Many papers cited as arXiv preprints were actually published in conferences and journal. Double check upper cases (e.g., [14] hamiltonian -> Hamiltonian). Some journals are abbreviated, some are not (e.g., [23], [27] vs others). Several citations display twice the year and/or the publisher (e.g., [13], [29], [48]). [10] and [46] cite the same paper. Some papers with many authors are cited usng "et al", some not ([33], [12] vs [7])
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are not well addressed
summary_of_recommendation: Overall, the proposed approach is interesting and may be valuable when deploying geometric fabrics on real robots. However, the contributions of the paper are rather incremental and mostly result from a combination of the ideas in [11] and [12]. Moreover, the presented architecture is very specific to the considered hand-and-arm system (especially concerning the design of the underlying geometric fabrics), and it is unclear how it may scale to other systems and tasks. A very large number of demonstrations was required for 2 out of the 3 considered tasks, thus hindering the general applicability of the proposed approach. Finally, several clarifications are required to make the paper self-contained.


--------
**Post-rebuttal comments**

I would like to thank the authors for their detailed answer to my questions and comments. I appreciate the authors' investment in the rebuttal. 

However, I think that the main issues of the paper remain after the rebuttal phase. The contributions as initially stated in the paper remain incremental, while I find that the additional contributions stated during the rebuttal are vague and barely reflected in the paper. Moreover, the issues about data efficiency have not been completely answered. Finally, my doubts about the effective dimensionality of the NGF remain as there is still some confusion about the role of the eigenspace for controlling the 16 DoFs of the hand. Although I agree that each eigengrasp controls the 16 DoFs of the hand, the benefit of using an eigenspace is that the final posture of the hand can be expressed as a function of the amplitudes along each of the eigengrasp direction. As 5 eigengrasps are used here, the posture of the hand should therefore be expressed as a function of the 5-dimensional amplitude vector (and not via the 16 original DoFs of the hand), which reduces the dimensionality of the problem.
Therefore, I maintain my initial evaluation.
issues: As mentioned in the "strengths and weaknesses" section, the following points should be clarified or improved in the revised manuscript:
- Novelty of the proposed approach;
- Applicability of the proposed approach to different systems;
- Very large number of demonstrations required to learn a task;
- Effective low dimensionality of the main geometric fabrics vs claim of learning high-dimensional policies;
- Clarifications on the content of the main paper;
- References and figures formatting.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper proposes a method for learning geometric fabrics from demonstrations for a high DoF hand-arm system. For this purpose the authors define dedicated subspaces suitable for the tasks to be solved and learn the policies as well as the priority metrics in these subspaces manner via behavior cloning. 
The authors demonstrate the capability of the proposed method to solve three robotic manipulation tasks, outperforming the chosen baselines in terms of task success rate, safety, imitation error and sample efficiency. 
strengths_and_weaknesses: Strength:
- The paper develops a method for learning geometric fabrics from demonstrations and demonstrates the capabilities of the resulting policy. 
- The results are compared to those of many baselines in well-chosen experiments and show superior performance in all the defined metrics.
- code is provided in the supplementary material

Weaknesses:
There remain some unclarities, as described in the issues section.


robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I recommend accepting the paper, since it proposes a promising approach for learning geometric fabrics from demonstrations. The capabilities of the method are demonstrated in well-chosen experiments and its superior performance w.r.t. many baselines is shown. 

issues: - I assume the training is done in an end-to-end manner, with the loss in eq. (9) in appendix D defined in the root space. But this is not stated clearly anywhere. I also think it would be beneficial to have more on the training in the main paper and not to defer it completely to the appendix, since it is an integral part for understanding the paper. 
- In line 241: I assume the '20 arbitrary object poses' used for evaluation where never seen during training, correct? I am unsure due to the ending of the sentence '... used for demonstrations.', which might refer to the 20 poses or to the table.
- In line 243: Why do you use the mean of the target poses seen in training? Why not use an arbitrary target pose just as it is done for the object poses? Also, do I assume correctly that the target pose is the same for all 20 evaluation runs?
- concerning RMPs: Are the metrics learned? If not, how are they chosen?
review_title: null

Meta-review: metareview: Reviewers agree that this is an interesting paper, however, they also identified a few issues with the paper:
- The relation and novelty w.r.t to the RMP paper seems unclear as both approaches seem to be very similar. 
- The comparison to the RMP approach is missing many details and one reviewer was questioning the poor performance of RMP as they performed well in other papers
- Additional baseline should be considered such as DMPs. DMPs have also been extended to neural network architectures, see [1].
- The data efficiency of the method has to be discussed and evaluated in more detail

These issues (as well as other reviewer comments) should be addressed carefully in the rebuttal. 

[1] https://papers.nips.cc/paper/2020/hash/354ac345fd8c6d7ef634d9a8e3d47b83-Abstract.html

Rebuttal Update: Unfortunately, the reviewers could not come to the same opinion on this paper even after a longer discussion period. 2 reviewers do see the following issues:
- in terms of novelty of the approach as the given architecture is a special case of an RMP and RMPs have already been "neuronalized". Further, they argue that the seen advantages of the architecture is a contribution of the original GF paper (which has no neural learning architecture in it). Even after a longer discussion with both reviewers, I can not follow their argument as the given paper is the first to put GFs in a neural architecture and show that it is more effective than a standard RMP architecture. I do see that as a significant contribution.  
- Furthermore the reviewers raised the issue that the approach is using eigengrasps to reduce the dimensionality of the problem. Even with eigengrasps the dimensionality of the problem is high higher then used in many other papers, so I also can not follow this argument.
- Another issue was raised in terms of data complexity as the approach needed 80 demonstrations which is, in comparison to other methods such as DMPs a quite high number. However, plain DMPs are a much simpler approach that would never scale to this complexity. The authors compared to Neural architectures of DMPs in the rebuttal, so I think this point is also addressed.

I agree that the contribution of the paper (differences to a standard RMP) should be made clearer in the paper and the different parts of the architecture should be ablated to see why we get such a performance difference (as part of future work). However, given that the paper offers an interesting contribution for data-driven motion generation and offers real-robot experiments of high complexity, I would actually side with the two positive reviews and would like to see the paper published. 




confidence: 3: The area chair is somewhat confident
Decision: Accept (Poster)