Neural Geometric Fabrics: Efficiently Learning High-Dimensional Policies from Demonstration
Authors: Mandy Xie
Ankur Handa
Stephen Tyree
Dieter Fox
Harish Ravichandar
Nathan D. Ratliff
Karl Van Wyk
Venue: CoRL 2022 Poster (2022)
Keywords: Imitation Learning
Dexterous Manipulation

Abstract: Learning dexterous manipulation policies for multi-fingered robots has been a long-standing challenge in robotics. Existing methods either limit themselves to highly constrained problems and smaller models to achieve extreme sample efficiency or sacrifice sample efficiency to gain capacity to solve more complex tasks with deep neural networks. In this work, we develop a structured approach to sample-efficient learning of dexterous manipulation skills from demonstrations by leveraging recent advances in robot motion generation and control. Specifically, our policy structure is induced by Geometric Fabrics - a recent framework that generalizes classical mechanical systems to allow for flexible design of expressive robot motions. To avoid the cumbersome manual design required by existing motion generators, we introduce Neural Geometric Fabric (NGF) - a framework that learns Geometric Fabric-based policies from data. NGF policies are provably stable and capable of encoding speed-invariant geometries of complex motions in multiple task spaces simultaneously. We demonstrate that NGFs can learn to perform a variety of dexterous manipulation tasks on a 23-DoF hand-arm physical robotic platform purely from demonstrations. Results from comprehensive comparative and ablative experiments show that NGF's structure and action spaces help learn acceleration-based policies that consistently outperform state-of-the-art baselines like Riemannian Motion Policies (RMPs), and other commonly used networks, such as feed-forward and recurrent neural networks. More importantly, we demonstrate that NGFs do not rely on often-used and expertly-designed operational-space controllers, promoting an advancement towards efficiently learning safe, stable, and high-dimensional controllers.

Neural Geometric Fabrics: Efficiently Learning High-Dimensional Policies from Demonstrations
Mandy Xie 2 Ankur Handa', Stephen Tyree', Dieter Fox
Harish Ravichandar", Nathan Ratlifr', Karl Van Wyk NVIDIA, Georgia Institute of Technology. 3 University of Washington.
Learning dexterous manipulation policies for multi-fingered robots has been a long-standing challenge in robotics. Existing methods either limit them- selves to highly constrained problems and smaller models to achieve extreme sam- ple efficiency or sacrifice sample efficiency to gain capacity to solve more complex tasks with deep neural networks. In this work, we develop structured approach to sample-efficient learning of dexterous manipulation skills from demonstrations by leveraging recent advances in robot motion generation and control. Specif- ically, our policy structure is induced by Geometric Fabrics a recent frame- work that generalizes classical mechanical systems to allow for flexible design expressive robot motions. To avoid the cumbersome manual design required by existing motion generators, a framework that learns Geometrie Fabric-based policies from data. NGF poli- cies are provably stable and capable of encoding speed-invariant geometries of complex motions in multiple task spaces simultaneously. We demonstrate that NGFs can learn perform variety of dexterous manipulation tasks on 23- DoF hand-arm physical robotic platform purely from demonstrations. Results from comprehensive comparative and ablative experiments show that NGF's struc- ture and action spaces help learn acceleration- based policies that consistently out- perform ofate-of-the-art baselines like Riemannian Motion Policies (RMPs), and other commonly used networks, such as feed-forward and recurrent neural net- works. More importantly, we demonstrate that NGFs do not rely on often- and designed operational-space controllers, promoting an advancement towards efficiently learning safe, stable, and high-dimensional controllers.
A Abstract:
introduce Neural Geometric Fabric (NGF)
Keywords:
Imitation Learning, Dexterous Manipulation
Introduction
Autonomous and robust dexterous manipulation capabilities could enable robots perform wide range of tasks, such as opening door and using tools, in a world built by and for humans. However, dexterous manipulation involving multi-finger hands continues to be a long-standing challenge in robotics, with human-level dexterity remaining a distant target. Indeed, there are numerous factors contributing to this state of affairs, such complex dynamics, high- tigh-dimensional action spaces, and the difficulties involved in designing bespoke controllers that do not generalize to either novel tasks or variations of the same task, such as new object poses or different targets.
Recent efforts robot learning have aimed to study and tackle the challenges that plague au- tonomous dexterous manipulation [1-11]. Specifically, recent learning- based techniques show promise by generating appropriate robot behavior for a variety of tasks without the need for hand- crafted motion policies. Most existing approaches can be divided into two broad areas: i) methods that utilize deep neural networks to learn highly parameterized, high-dimennsionl policies that solve complex dexterous manipulation tasks with minimal manual design, but the cost of low sam- ple efficiency and lack of theoretical guarantees [6-8, 12-15], and ii) methods that utilize highly structured models with significantly fewer parameters that can achieve extreme data efficiency, but are either limited to highly constrained problems or depend on expertly-designed operational space controllers [16-26].
In this work, we attempt to combine the best of both high-capacity models and samplc-efficient structures in order to learn high-dimensional dexterous manipulation policies for physical hand-
"manxie@gatech.edd
6th Conference on Robot Learning (CoRI. 2022), Auckland, New Zealand.

---

target
Task Sugar
Push Coffee Can
target
Task Tray
Figure I: NGFs can learn to control joints a hand-arm system to perform tasks, such as (a) grasping and lifting a box, (b) touching a target point before pushing can over it, and (c) extracting a toolbox tray.
arm system purely from a limited number of demonstrations. We are concerned with learning poli- cies that directly and simultancously coordinate all the joint-space degrees of freedom both the arm and the hand and can generate stable and smooth motions for tasks requiring dexterity, such as grasping and manipulating an object with a multi- multi-finggeed robotic hand.
Inspired by the growing evidence in support of structured policies for manipulation (e.g.. [27, 28)), we propose an algorithm to learn highly-structured and generalizable policies for dexterous manip- ulation from limited number of demonstrations. Specifically, we leverage the structure imposed by Geometric Fabrics [29], a recent framework that generalizes classical geometric control (e.g., operational space control) using tools from differential geometry, facilitate stable and flexible robot motion generation [30]. Geometric fabrics are stable second-order dynamical systems, which encode behaviors speed-invariany paths (via geometries) and generate acceleration actions to gen- erate intended motions. Further, learning to encode dynamical systems via acceleration known to be a significantly beneficial form of inductive bias in policy learning [31]. However, designing these fabrics for complex dexterous manipulation tasks requires considerable expertise and effort.
To circumvent the need for significant expertise and painstaking human effort in designing Geo- metric Fabrics for dexterous manipulation tasks, we introduce Neural Geometric Fabrics (NGF), a framework to efficiently learn components of Geometric Fabrics, such subtask maps, geometries, and priority metrics. We discuss details of our architecture and learning pipeline in Section
The primary contributions of this paper are: 1) novel framework to learn Coommttrc-Fabrics-aee policies directly from demonstrations, removing the need for manual design and expertise; 2) a thorough investigation of the benefits of additional structure imposed by Geometric Fabrics over existing approaches when learning dexterous manipulation skills from demonstrations; and 3) the best of our knowledge, this is the first framework to train and deploy policies that directly control all joints the highly-actuated arm-hand system in the real world without relying on manually- designed operational-space controllers in elevated actions spaces.
We evaluate the effectiveness of our approach on 23-DoF physical hand-arm robot platform in the real world across three different tasks (see Fig 1). and compare its performance against several baselines inspired by the state-of-the-art in learning from demonstration. Our results show that the policies learned with our structured approach consistently outperform baseline methods with respect to metrics measuring task success, safety of deployment, and sample efficiency.
2 Related Work
A variety of approaches have attempted to tackle the challenges of learning dexterous manipulation polices. Here, we discuss how our approach relates a few different categories of existing methods.
Deep neural network based approaches such as deep Reinforcement Learning (RL) methods [7- 9, 14, 32] have demonstrated impressive capabilities. However, most RL algorithms suffer from poor sample efficiency. On the other hand, Imitation Learning (IL) improves sample efficiency by leveraging expert demonstrations [1, 33). However, pure IL methods are often limited to non- dexterous manipulation [2, 26, 34, 35]. Recent approaches have combined IL with RL efforts to improve data efficiency in learning complex dexterous manipulation skills [3-6, 12, 36). However, these methods are often limited to simulation as collecting exploratory data can be expensive, time- intensive, and potentially damaging to the hardware in real-world robotic systems. Given these

---

observations, it is critical to develop robot learning frameworks that can learn to generalize from less data. We demonstrate that NGF can learn complex dexterous manipulation skills directly on hardware purely via imitation, without expertly-designed operational space controllers.
Exploiting structures in robotic manipulation problems has shown to improve data efficiency in policy learning even though structured policy parameterizations are more restricted in representa- tion power than their non-structured counterparts [37]. This is due to the fact that, if the restrictions provide appropriate inductive biases for the task of interest, structured approaches provide numer- ous advantages, such as provable stability, safety guarantees, and improved efficiency, without ad- versely impacting expressivity. Indeed, there are several successful examples of learning structured policies. The approaches in [38-42] leverage structures from control theory, where learned poli- cies meet optimal control assumptions to provide stability guarantees. Dynamical system based approaches [16-26] learn policies by imposing structures or constraints that enforce stability and convergence while imitating the demonstrations. Recently, it was shown that Riemannian Motion Policies (RMPs) [27, 28, 43, 44] can encode the geometric structure of manipulation tasks using structured second-order dynamical systems. One can view the proposed NGFs as specialized ver- sions of RMPs that are stable and capable of encoding more complex behaviors as geometries, providing more appropriate inductive bias for dexterous manipulation. See Appendix L for more detailed comparison between NGF and RMP.
The choice of action spaces plays significant role in learning manipulation policies [37], with the most popular choice being the end-effector pose space [2. 26, 32. 33] as it is more directly relevant to the task and can generalize across different robot kinematics [45, 46). These approaches rely on an operational space controller to map these desired signals into the joint space. However, is not trivial to design such a controller for highly kinematically-redundda robots, such as multi funti-finggred hands. In contrast, our method does not require any hand-crafted operational-space controllers as leverages the robot's kinematic model to directly learn in multiple actions spaces simultaneously.
3 Background: Geometric Fabrics
Geometric fabrics are the latest incarnation of research in robot motion generation and control as defined by second order equations. Geometric fabrics build upon the more general, intuitive, and flexible aspects of Rimennian Motion Policies (RMPs), but retain the stability guarantees offered by classical mechanical systems (which are often leveraged in control as geometric operational- space control.) In fact, Geometric Fabrics generalize classical mechanical systems and have been shown to outperform RMPs [29]. Geometrie fabrics provide formal mathematical framework for building control policies and primarily encode behavior as a set of nonlinear geometries of paths. This geometrically consistent behavior known as the nominal behavior of the system and acts as an optimization medium over which additional policies derived from potential functions and damping can be applied to further shift the resulting behavior. Policies of this design are guaranteed to be stable, i.e., the system comes to rest at the minimum of the net potential acting on the system.
3.1 Nonlinear Geometries
A generalized nonlinear geometry an acceleration policy x xxx x)
where is homogeneous
of degree 2 (HD2), meaning for any X > 0 we have π (x, Ax) X² (x, x). The HD2 property ensures that every integral curve starting from a given task-space position X. with velocity X, 1ÿ will follow the same path. The geometric consistency property turns æ into geometry of paths. From a imitation learning perspective, this additional structure promotes learning speed-ine invariant paths through space instead of trajectories. which are sensitive to the speed of travel. We postulate the added structure will be beneficial for sample-efficiency in policy learning.
3.2 Geometric Fabrics: encode nominal behaviors
A Geometric Fabric is system that evolves according to M(q. q)å + f(q. 4) 0 in root coor- dinates q, where q -M is a nonlinear geometry that defines nominal behavior and M is the priority metric (we drop function variables for notational simplicity). As discussed in Theorem IV.5 of [29], we can energize this Geometric Fabric conserve energy: q = -P. [M ff. where
P. M 3 I I- - a weirie-weighted projection matrix with v = M b and = v/llv|],
3

---

which projects acceleration to the orthogonal space of velocity, hence conserves energy by perform- ing no work. The energized fabric defines a nominal behavior independent of specific task.
3.3 Optimize a potential: solve a task
The nominal behavior can be pushed away from by potential function to achieve task goals, and damping dissipation ensures convergence. The desired equation of motion of the system then be- M
comes: du -P [M- 0 (()) Bq. where (b is potential function, and B is a positive definite damping matrix. The potential function induces kinetic energy, resulting in motion, and together with damping, drives the system towards the potential's 's local minima, where motion ultimately comes rest at the minima of the potential function, hence its optimization.
3.4 Policy Design with Geometric Fabries
For robot manipulators. a desired behavior may involve coordinated motion of different parts. Ge- ometries can be added to spaces of a transform tree [47] for the modular design of composite be- haviors. We can construct a transform tree with root node in the configuration space C (typically joint space) of the robot, and add leaf node in each subtask space Ty in which the task is more convenient to be described. For instance, a goal -ealhing task can be defined in the 3-D Euclidean space describing the position of the end-effector relation to the goal. We denote subtask spaces {T&) where K is the number of subtasks. Let Φx : C + Ti be the mapping from the config- uration space to subtask space Ty. An example of dR the forward kinematic mapping, where the subtask space To is the end-effecte frame. We then populate each node with a priority metric and geometry pair, [M,, xx), which are pulled-back and combined at the root, resulting in desired configuration space policy qa कz(q. q). In the case where we only have an end-effector space, solving for e is equivalent to a pseudo-inverse as described in [48]. We include additional details about composing policies from sub-task spaces in Appendix A.
4 Neural Geometric Fabrics
In this section, we introduce our NGF policy architecture and its learnable components. We illustrate our design process in Fig, 2, and elaborate each step in the sections below.
4.1 Define subtask spaces
We begin by defining two subtask spaces: (1) A 3-D Euclidean space describing the position of the palm in relation to the object, which is stacked with an Eigenspace of the hand. This map is defined as Φ1 : q ++ [fk] (q) x pca" (q))² E 8x1 where fk(q) the forward kinematics mapping to palm point and x. is the object position. pca(q) is linear map of the hand, inspired by the concept of Eigengrasp [49]. where we first apply Principle Component Analysis (PCA) to the 16- DoF hand joints data, which reduces the 16-DoF joint space to the 5-DoF Eigenspace while the first five principle components account for more than 95% of the variance. This stacked space directly shares information between the arm and hand, allowing them to coordinate properly. (2) The full configuration space of the robot, d2 q + q E 23 The policy in this space additionally shapes the configuration space behavior, filling in the nullspace of the preceding space.
4.2 Define learnable components in subtask spaces
Geometric policies: A priority metric and geometry pair in the stacked space Φ1 is defined as [M1 encoding object-reaching and coordinated palm-finger behavior. Another priority metric and geometry pair [M,, π2] is defined in the configuration space, Φ2. that serves as residual policy, shaping the configuration space behavior and ensuring the combined Geometric Fabric is full rank.
Potential and damping: We additionally add another priority metric and acceleration based poten- tial policy [M,, y (Ty is purely function of position) in the stacked space Φ1 to: 1) maintain bounded system energy levels while 2) induce motion (energized geometries cannot induce motion by themselves when at rest), and 3) optimize the corresponding potential function. Finally, a strictly positive damping scalar b in the configuration space, Φ2, is also learned and applied that removes energy from the system, ensuring convergence to the minima of the potential function.

---

Neural Geometric Fabric
Learnable components
Geometric Fabric
Metric Network
Push-forward
Configuration space
Forward Mapping
spaces
(1))
(g 5
(g)
Auxiliary Network
Gonmatric politins
*grccibaL
M. M.
M,
likk (xa)
loooI
J, (c. M.J.J.J Combined geometric policy
B,4s
Poteriaa poicy
M, My(x;; )
0,, (x)(1)
M Ff
M.n.)
Demping Network
Pull-back
(q.
Energized toroed
comperents
Complete form: q = P.
M
6g
Blq.
Enerpization
Potental
Domping
Figure 2: We parameterize NGFs with neural networks and construct them follows: (1) define each subtask space with forward mapping Φi that maps coordinates q in the configuration space into coordinates X₁ in the k-th subtask space, (2) define geometric policy pair [Ms; T&. in each subtask space Xs. e.g., = 2, and define a potential policy pair [My; Ty] in the desired sub- task space, e.g., X1. (3) pull-back the geometric and potential policy pairs into the configuration space, (4) combine the geometric policies via a metric-weighted average, (5) energize the combined policy (project orthogonal to the direction of motion with Pe) create collection of energy- preserving paths (the Geometric Fabric), and (6) force the Geometric Fabric with potential defined by ,M,, y and damp via 6 applied along q. which ensures convergence to the potential's minima. Note that we parameterize the geometric policy pairs [M,, T the potential policy pair [M,, Ty}, and the damping scalar b with network networks and learn them from data. We also define d% with neural networks desired. For notation simplicity, we use subscript x to represent 1, 2, and
All learnable components, including M., n.] (r, is constructed with h, and 1,2, and the damping scalar b, are defined with neural networks. To ensure the priority metrics are positive defi nite, follow the architecture described in [28]. See Appendix B for detailed parameterizations,
4.3 Complete form
Using the pullback and combination operations defined in Appendix A, the two geometric policies and potential-based policy are combined at the root along with damping and energization. With the total metric, M M, + M, M,, total geometric force, f = fi f2, and potential force fy. the desired NGF acceleration takes the form q, P M M M- Mf bq, where P. (I -I-qq") energizes the fabric with squared velocity norm energy (see equation (10) of [29] concerning this energization), and IR is a damper. d time integrated to produce qa and < the command trajectory for the robot to follow (see algorithms in Appendix C).
~
Learning Policies from Demonstrations
Given the policy architecture Section 4, here we describe how we train NGF--based policies using demonstrations to solve manipulation tasks on a 23-DoF hand-arm system system (7-DoF arm and 16-DoF multi-iingered hand). We used the DexPilot teleoperation system [50] collect task-solving demonstrations, and trained policy to replicate the demonstrated joint position trajectories.

---

5.1 Problem Statement
Consider deterministic discrete- time system with transition model s4+1 = [(t, at) where s, I is the state, and a, E R is the action of the system, and f : I² x R + IR" is the system transition function. We want learn an NGF policy T& : s + (T) II { A E }}. given N trajectory demonstrations (T(*) Ji=1- Each demonstration is defined as a sequence of states, denoted 7(1) ;= {s ( Dr= where T; is the horizon for the ith demonstration. Note, the system transition model we refer to here simply Euler integration defined by Eq. (7) in Appendix D.
5.2 Policy Optimization
To mitigate the distribution shift issue involved in behavior cloning, we formulate the policy learning problem as multi-step prediction error optimization problem, in which we learn policy that can reproduce the demonstrated behavior by minimizing the deviation between the demonstrated trajectory and trajectory produced by a learned policy under the system transition function f. That is, given transition function 8t+1 := [*t, TT((s) (+)) for each policy, we can rollout a policy trajectory and perform poopppationnttrrrooooeo
timmttime [51, 52] with the Adam optimizer [53] to improve the policy parameters. Importantly, the states here are the commanded states during demonstration.
More details on policy optimization and the exact loss function are included in Appendix D.
6 Experiments
We studied the effect of i) policy structure and ii) the choice of action space by comparing a variety of policy classes across three different tasks (see Fig 1). The first task illustrates prehensile manipulation by requiring the robot to grasp and lift a randomly placed sugar box from the table. The second task illustrates reaching, collision avoidance, and non-prehensile manipulation by re- quiring the robot to touch target point on the table while avoiding collision with randomly placed coffee can and then push the can over to the target. The third task illustrates constrained prehensile manipulation by requiring the robot to precisely maneuver its fingers through a tool tray handle and extract it from a tool box while avoiding undesirable collisions before placing it onto the table.
Tasks:
We compared the performance of NGF against the following baselines: a fully-connected Neural Network policy (NN), Riemannian Motion Policy (RMP), a recurrent neural network policy with Long Short-Term Memory units (LSTM), and an End-Effector Neural Network policy coupled with manually-designer
Baselines:
Geometric Fabric for operational space control (EEF-GF). We chose these specific baselines as they represent current practices in imitation learning for manipulation with re- spect to the choices of policy structure and action space. We provide additional details and justifica- tions in Appendix E, and details about the experiment setup and data collection in Appendix F.
6.1 Evaluation Procedure
We trained all policies using the procedure described in Section 5 and deployed them on a physical robot platform to empirically measure their performance in representing the desired behavior im- plicit in the demonstrations. To be successful, the policies must simultaneously encode approach and grasping behavior by directly coordinating finger, palm, and arm motions for each task.
For each of the first two tasks involving prehensile and non-prehensile manipulation, we trained one instance of each policy class (NN, RMP, LSTM, EEF-GF, and NGF) on 80 demonstrations. We then evaluated each trained policy's 'ss ability to generalize to 20 arbitrary object poses on the surface of the table used for demonstrations. Each policy received the object pose estimated by CosyPose [54], target pose equivalent to the mean pose in the training data, and initial robot configuration as inputs. For the third task involving tool tray retrieval, we trained one instance of each policy class on 6 demonstrations, in which the toolbox is positioned at 6 different orientations [10. 30, 50, 70, 90, 110) (deg.). We evaluated each trained policy's s ability to retrieve the tool tray with the toolbox positioned at 9 different orientations |-10, 0, 20, 40, 60, 80, 100, 120, 130] (deg.). Each policy received the toolbox orientation and the initial robot configuration as inputs.
We quantified policy performance by evaluating both task success rate and the safe deploy- ment rate. To compute safe deployment rate and prevent hardware damage, we used each trained policy to generate open-loop trajectories for horizon of 13 seconds for Task 1, and 30 seconds
Metrics:

---

for Tasks 2 and 3. We then visualized the robot enacting these trajectories using PyBullet [55] to determine its safety for deployment. We consider trajectory be safe if it does not excessively throw the robot into, through, or below the table, and does not lead to violent arm movements. We deployed all safe trajectories on the physical robot using PD controller. To compute task success rate, we considered an execution to be successful only if the following criteria are met within the deployment horizon. Task 1: the robot grasps and lifts the sugar box from the table; Task 2: the robot first touches the target point without colliding into the coffee can and then moves the can over to the target location; Task 3: the robot grasps and lifts the tool tray out the toolbox. addition to these two metrics, also measured the imitation error of each policy (defined in Eq. (12) of Appendix G) to evaluate sample efficiency.
6.2 worldworld Task Performance
Task Success Rate: The following key observations can be made from the results reported in Fig. 3a: (i) Most strikingly, NGF consistently offered the highest level performance against all baselines and outperformed the best baseline by 40% to 100%, indicating the benefits of our specific architecture and our use multiple action spaces. (ii) EEF-GF had the second highest level of per- formance by matching other baselines the coffee and tool tray tasks and outperforming them on the sugar box task. EEF-GF's good performance is explained by the fact that policies trained with elevated action spaces (in this case, palm pose action) known to be easier learn. However, it is important to note that EEF-GF enjoys the henefits manually-tuned Geometric Fabric-based oper- ational space controller, and yet preforms worse than NGF which does require such controller. This result demonstrates the benefits of learning high-dimensional structured policies directly from data. (iii) Both second-order baselines, RMP and NN, surprisingly offer relatively same level of task success rates despite the added taskmap structure of the RMP which aims to introduce beneficial inductive biases. Although less clear, RMP and NN seemed offer marginal improvements over LSTM, suggesting that second-order policies' ability to encode complex motions is comparable to that of well-established recurrent structure commonly found in the literature.
100
NN
100
RMP
Safe Deploy Rate %
FINA-G (MURE)
Success Rate
60
NGE COURS:
40
40
Sugar Hox
Coffee Can
Tnolbox
0 Sugar Box
Colfee Can
Taolhox
(a)
(b)
Figure 3: Policy performance all three tasks based on (a) percentage of deployments that were successful, and (b) percentage of deployments that were safe to deploy on the real robot.
Safe Deploy Rate:
We report the safe deployment rates for each policy and task Fig. 3b. Crit- ically, both NGF and EEF-GF were the safest deploy across all tasks with 100% deploy rates, indicating that Geometric Fabrics-induced structure helps generate smooth and safe motions. Next, RMP had consistently higher deployment rates over NN, indicating that its taskmap structure helps improve safety. Finally, LSTM was comparable to RMP in generating safe trajectories.
6.3 Analysis on Sample Efficiency
We also studied the sample efficiency of each policy when learning to perform the first task. Imitation Error:
Imitation error captures the difference in trajectories generated by the learned policy and trajectories from the unseen data. We trained each policy sets of demonstrations with sizes [10, 30, 60), and for every 10 epochs, computed imitation error on a set of 20 held-out demonstrations. After training for 2000 epochs, we selected the best model based on the imitation error for each policy over 5 random initialization seeds. We report the mean and standard deviation of the imitation error for each policy across 5 random seeds in Fig. 4a. We note that NGF is the
7

---

most sumple-efficieen as it results in the lowest mean imitation error (and smallest variance) across all quantities of demonstrations.
0.150
Imitation Eiro vs Dataset Size
100
Success Rate vs Duraset Size
100
Safe Deploy Rate vs Dataset Size
NN
Initetion Eator m
RMP
LSTM
LSTM
Safe Deploy Rate
BEF- NGP (DURS)
Success Rate
ERF-GI (00
0.100
(OURS)
60
40
40
0.075
0.050
Number Demonstrations
30
Number Demonstration
0-
Number Demonstrations
60
(c)
Sample efficiency analysis based on (a) imitation error of each policy (solid lines indicate the mean and the shaded area show mean standard deviation, over the 5 random seeds). (b) task success rate, and (c) safe deployment rate on physical robot.
Figure 4:
Task Performance and Safety: Following the procedure in Section 6.1, we report how each policy's task success rate and safe deployment rate change with dataset size in Figs. 4b and 4c, respectfully. We note that real-world performance somewhat correlates with imitation error, but not strictly as observed in [2]. Similar to the results in Section 6.2, NGF consistently outperforms all the baselines across all dataset sizes (except for EEF-GF's better safe deployment rate with only 10 demonstra- tions, perhaps due its manually-designee controller). Overall, the superior performance of NGF in all three metrics indicate that the additional structure imposed by Geometric Fabrics helps improve sample efficiency in policy learning. We provide an extended discussion Appendix G.I.
Final Remarks
Discussion The mixture of near pear-perfect deploy rates and significantly higher task success rates of NGF against all baselines demonstrates the structural power of Geometric Fabrics. We find that EEF-GF is also very viable architecture which demonstrates how the elevated pose action space when combined with an analytically-derived Geometric Fabric can improve policy learning. Our findings suggest that learning entirely new control structures with multiple action spaces beyond end-effector poses is possible with Geometric Fabrics. Although the policies were deployed open- loop, this is not intrinsic limitation of NGFs. To illustrate its reactivity, policy on the coffee can task in closed-loop. We note that NGF successfully adapted and performed the task, even when pushed the coffee can during execution (see Appendix H for more details).
deployed the NGF
Limitations Due to the high cost of running real-world experiments, we partially tested at most three random seeds per policy class per task and selected the best seed for full assessment. We acknowledge that experimental and policy variance could produce artifacts in the results. However, the combination of: 1) screening the best policy for each class and task, 2) the low imitation error and associated variance of NGF (see Fig. 4a), and 3) the consistency of NGF excelling across tasks makes it very unlikely that any of the baselines offer better performance in practice. We suspect that the performance of the EEF-GF policy could be improved by including the underlying Geometric Fabric policy inside the training loop. Finally, we deployed the policies open-loop for two primary reasons: 1) object pose detection becomes unreliable during heavy occlusion, and 2) we could as- sess the safety of the policy's 'ss trajectory before deployment, protecting our hardware. We note that training with much larger and diverse datasets will likely enable reliable closed-loop control.
Future work Our work opens up number of avenues for future work, including 1) learning NGF policies from significantly larger and more diverse datasets, 2) learning NGFs that can process raw sensory streams including camera data, and 3) investigating the benefits of NGF for reinforcement learning. We believe that NGF- based learning frameworks have the potential to solve a wide variety of manipulation tasks in closed-loop, enabling of state (e.g., pose) and towards processing raw proprioceptive, tactile, and visual feedback a structured manner.
move away from simplified representations
8

---

Acknowledgments
This research was supported by NVIDIA Research.

---

References
[1] H. Ravichandar, A. S. Polydoros, S. Chernova, and A. Billard. Recent advances in robot learning from demonstration. Annual Review of Control, Robotics, and Autonomous Systems, 3:297-330, 2020.
[2] A. Mandlekar, D. Xu. J. Wong, S. Nasiriany, C. Wang, R. Kulkarni, L. Fei-Fei, S. Savarese, Y. Zhu, and R. Martin-Martin. What matters learning from offline human demonstrations for robot manipulation. arXiv preprint arXiv:2108, 03298, 2021.
[3] Y. Qin, Y.-H. Wu, S. Liu, H. Jiang. R. Yang, Y. Fu, and X. Wang. Dexmv: Imitation learning for dexterous manipulation from human videos. arXiv preprint arXiv:2 202.0008.77777 2021.
[4] R. Jeong, J. T. Springenberg. J. Kay, D. Zheng, Y. Zhou, A. Galashov, N. Heess, and F. Nori. Learning dexterous manipulation from suboptimal experts. arXiv preprint arXi::2010.08387 2020.
[5] I. Radosavovic, X. Wang. L. Pinto, and J. Malik. State-only imitation learning for dexterous manipulation. arXiv preprint arXiv:2004. 046500, 2020.
[6] A. Rajeswaran, V. Kumar, A. Gupta, G. Vezzani, J. Schulman, E. Todorov, and S. Levine. Learning complex dexterous manipulation with deep reinforcement learning and demonstra- tions. arXiv preprint arXiv: aee:: 10087, 2017.
[7] I. Popov, N. Heess, T. Lillicrap, R. Hafner, G. Barth-Maron, M. Vecerik, T. Lampe, Y. Tassa, T. Erez, and M. Riedmiller. Data-efficient deep reinforcement learning for dexterous manipu- lation. arXiv preprint arXiv:1704.0300 2017.
[8] O. M. Andrychowicz, B. Baker, M. Chociej. R. Jozefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, Ray, et al. Learning dexterous in-hand manipulation. The International Journal of Robotics Research, 39(1):3 2020.
[9] A. Nagabandi, K. Konolige, S. Levine, and V. Kumar. Deep dynamics models for learning dexterous manipulation. In Conference on Robot Learning, pages 1101-1112. PMLR, 2020.
[10] A. Gupta, C. Eppner, S. Levine, and P. Abbeel. Learning dexterous manipulation for a soft robotic hand from human demonstrations. In 2016 IEEF/RSI International Conference on Intelligent Robots and Systems (IROS), pages 3786-3793. IEEE, 2016.
[11] P. Ruppel and J. Zhang. Learning object manipulation with dexterous hand-arm systems from human demonstration. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5417-5424. IEEE, 2020.
[12] T. Chen, J. Xu, and P. Agrawal. A system for general in-hand object re-orientation. In Confer- ence on Robot Learning, pages 297-307. PMLR, 2022.
[13] I. Akkaya, M. Andrychowicz, M. Chociej, M. Litwin, B. McGrew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribas, al. Solving rubik's cube with robot hand. arXiv preprint arXiv: 19000:191. 2019.
[14] W. Huang. I. Mordatch, P. Abbeel, and D. Pathak. Generalization in dexterous manipulation via geometry-aware multi-task learning. arXiv preprint arXiv:2111.03000 2021.
[15] H. Zhu, A. Gupta, A. Rajeswaran, S. Levine, and V. Kumar. Dexterous manipulation with deep reinforcement learning: Efficient, general, and low-cost. In 2019 International Conference Robotics and Automation (ICRA), pages 3651-3657. IEEE, 2019.
[16] P. Pastor, H. Hoffmann, T. Asfour, and S. Schaal. Learning and generalization of motor skills by learning from demonstration. In 2009 IEEE International Conference on Robotics and Automation, pages 763-768. IEEE, 2009.
[17] A. J. Ijspeert, J. Nakanishi, H. Hoffmann, P. Pastor, and S. Schaal. Dynamical movement primitives: learning attractor models for motor behaviors. Neural computation, 25(2):328- 373, 2013.
10

---

[18] S. Schaal, J. Peters, J. Nakanishi, and A. Ijspeert. Learning movement primitives. In Robotics research. the eleventh international symposium, pages 561-572. Springer, 2005.
[19] G. Maeda, M. Ewerton, T. Osa, B. Busch, and J. Peters. Active incremental learning of robot movement primitives. In Conference on Robot Learning, pages 37-46. PMLR, 2017.
[20] A. Paraschos, C. Daniel, J. Peters, G. Neumann, et al. Probabilistic movement primitives. Advances in neural information processing systems, 2013.
[21] G. Maeda, G. Neumann, M. Ewerton, R. Lioutikov, O. Kroemer, and Peters. Proba- bilistic movement primitives for coordination of multiple human-roke robot collaborative tasks. Autonomous Robots, (1335593--1,, 2017.
[22] S. M. Khansari-Zadeh and A. Billard. Learning stable nonlinear dynamical systems with gaus- sian mixture models. IEEE Transactions on Robotics, 27(5):943-957, 2011.
[23] S. M. Khansari- and A. Billard. Learning control Iyарunov function to ensure stability of dynamical system- based robot reaching motions. Robotics and Autonomous Systems, 62(6): 752-765, 2014.
[24] N. Figueroa and A. Billard. ory, learning, and experiments. 027836492110309952 2022.
Locally active globally stable dynamical systems: The- The International Journal of Robotics Research, page
[25] H. Ravichandar and A. Dani. Learning position and orientation dynamics from demonstrations via contraction analysis. Autonomous Robots, +3(4):897-913 2019.
[26] H. Ravichandar, I. Salehi, and A. Dani. Learning partially contracting dynamical systems from demonstrations. In Conference on Robot Learning, pages 369-378. PMLR, 2017.
[27] A. Li, C.-A. Cheng, M. A. Rana, M. Xie, K. Van Wyk, N. Ratliff, and B. Boots Rmp2: A structured composable policy class for robot learning. arXiv preprint 20w:21033.592... 2021.
[28] M. A. Rana, A. Li, H. Ravichandar, M. Mukadam, S. Chernova, D. Fox, B. Boots, and N. Ratliff. Learning reactive motion policies multiple task spaces from human demon- strations. In Conference on Robot Learning, pages 1457-1468. PMLR, 2020.
[29] K. Van Wyk, M. Xie, A. Li, M. A. Rana, B. Babich, B. Peele, Q. Wan, I. Akinola, B. Sun- daralingam, D. Fox, et al. Geometric fabrics: Generalizing classical mechanics to capture the physics of behavior. IEEE Robotics and Automation Letters, 2022.
[30] N. D. Ratliff, K. Van Wyk, M. Xie, A. Li, and M. A. Rana. Generalized nonlinear and finsler geometry for roboties. In 202/ IEEE International Conference Robotics and Automation (ICRA), pages 10206-10212, 2021. doi: 1090ICRA440662221..55555
[31] N. Gruver, M. A. Finzi, S. D. Stanton, and A. G. Wilson. Deconstructing the inductive biases of hamiltonian neural networks. In International Conference on Learning Representations, 2022.
[32] A. Gupta, J. Yu, Z. Zhao, V. Kumar, A. Rovinsky, K. Xu, T. Devlin, and S. Levine. Reset- free reinforcement learning via multi-task learning: Learning dexterous manipulation behav- iors without human intervention. 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 6664 6671, 2021.
[33] T. Osa, J. Pajarinen, G. Neumann, J. A. Bagnell, P. Abbeel, and Peters. An algorithmic perspective on imitation learning. arXiv preprint arXiv:1811.06711 2018.
[34] D. Pathak, P. Mahmoudich, G. Luo, P. Agrawal, D. Chen, Y. Shentu, E. Shelhamer, J. Malik, A. A. Efros, and T. Darrell. Zero-shot visual imitation. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 2050-2053, 2053, 2018.
[35] A. O. Ly and M. A. Akhlouñi. Learning to drive by imitation: an overview of deep behavior cloning methods. IEEE Transactions Intelligent Vehicles, 2020
11

---

[36] D. Jain, A. Li, S. Singhal, A. Rajeswaran, V. Kumar. and E. Todorov. Learning deep visuomo- tor policies for dexterous hand manipulation. In 2019 International Conference on Robotics and Automation (ICRA), pages 3636-3643. IEFE, 2019.
[37] O. Kroemer, S. Niekum, and G. Konidaris. A review of robot learning for manipulation: Challenges, representations, and algorithms. Mach. Learn. Res., 22:30-1, 2021.
[38] G. Dalal, Dvijotham, M. Vecerik, Hester, C. Paduraru, and Tassa. Safe exploration in continuous action spaces. arXiv preprint arXiv: 1801.0877,
[39] Y. Chow, O. Nachum, A. Faust, E. Duenez- Guzman, and M. Ghavamzadeh. Lyapunov-based safe policy optimization for continuous control. arXiv preprint arXiv: 10031, 2019.
[40] Choi, Castaneda, C. Tomlin, and K. Sreenath. Reinforcement learning for safety- critical control under model uncertainty, using control Iyарunоv functions and control barrier functions. arXiv preprini wwxiv:2004.075880 2020.
[41] S. Levine and P. Abbeel. Learning neural network policies with guided policy search under unknown dynamics. In NIPS, volume 27, pages 1071-1079. Citeseer, 2014.
[42] S. Levine, N. Wagener, and P. Abbeel. Learning contact-rich manipulation skills with guided policy search (2015). arXiv preprint arXiv:/501.003 0000/ 2015.
[43] M. Mukadam, C.-A. Cheng. D. Fox, B. Boots, and N. Ratliff. Riemannian motion policy fusion through learnable lyapunov function reshaping. In Conference on robot learning, pages 204-219. PMLR, 2020.
[44] X. Meng, N. Ratliff, Y. Xiang, and D. Fox. Neural autonomous navigation with riemannian motion policy. In 2019 International Conference on Robotics and Automation (ICRA), pages 9860-8866. IEEE, 2019.
[45] M. Mason. Compliance and force control for computer controlled manipulators. IEEE Transactions on Systems, Man, and Cybernetics, 1 11(6):418 432, 1981.
[46] D. H. Ballard. Task frames in robot manipulation. In AAAI, volume 19, page 109, 1984.
[47] C.-A. Cheng, M. Mukadam, J. Issac, S. Birchfield, D. Fox, B. Boots, and N. Ratliff. RMPflow: computational graph for automatic motion policy generation. In The 13th International Workshop the Algorithmic Foundations of Robotics, 2018.
[48] A. Li, C.-A. Cheng, M. A. Rana, M. Xie, K. Van Wyk, N. Ratliff, and B. Boots. RMP2: A structured composable policy class for robot learning. Robotics: Science and Systems, 2021.
[49] M. Ciocarlie, C. Goldfeder, and P. Allen. Dexterous grasping via eigengrasps: A low- dimensional approach to high-complexity problem. In Robotics: Science and systems ma- nipulation workshop-sensingg and adapting to the real world. Citeseer, 2007.
[50] A. Handa, K. Van Wyk, W. Yang. Liang, Y.-W. Chao, Q. Wan, S. Birchfield, N. Ratliff, and D. Fox. Dexpilot: Vision-based teleoperation of dexterous robotic hand-arm system. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 9164-9170. IEFE, 2020.
[51] P. Werbos. Backpropagation through time: what it does and how to do it. Proceedings of the IEEE, 18110):150--150, 1990.
[52] Langford, R. Salakhutdinov, and Zhang. Learning nonlinear dynamic models. In Pro- ceedings of the 26th Annual International Conference on Machine Learning, pages 593-600, 2009.
[53] D. P. Kingma and J. Ba. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.690 2014.
[54] Y. Labbé, J. Carpentier, M. Aubry, and J. Sivie. Cosypose: Consistent multi-view multi-object 6d pose estimation. In European Conference on Computer Vision, pages 574-591. Springer, 2020.
12

---

[55] E. Coumans and Y. Bai. Pybullet, python module for physics simulation for games, robotics and machine learning. attp://pbbullet... org. 2016-2021.
[56] K. Greff, R. K. Srivastava, J. Koutník, B. R. Steunebrink, and J. Schmidhuber, Lstm: A search space odyssey. IEEE Transactions on Neural Networks and Learning Systems, 28(10):2222- 2232, 2017.
[57] Florennce, L. Manuelli, and R. Tedrake. Self-supervised correspondence in visuomotor policy learning. IEEE Robotics and Automation Letters, 5(2):492 499, 2020. 2956365.
[58] T. Zhang, Z. McCarthy, O. Jow, D. Lee, X. Chen, K. Goldberg, and P. Abbeel. Deep im- itation learning for complex manipulation tasks from virtual reality teleoperation. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 1oi: 1091100109/1CRR2201888 2464444449..1884442 8461249.
2018.
[59] Bahl, M. Mukadam, A. Gupta, and D. Pathak. Neural dynamic policies for end-to-end sensorimotor learning. Advances in Neural Information Processing Systems, 33:5058-5069, 2020.
13


Reviews:
[{"review_id": "oR0qDe99gH_", "reviewer": "Reviewer_oe7Z", "summary": "The paper proposes a method for learning geometric fabrics from demonstrations for a high DoF hand-arm system. For this purpose the authors define dedicated subspaces suitable for the tasks to be solved and learn the policies as well as the priority metrics in these subspaces manner via behavior cloning. \nThe authors demonstrate the capability of the proposed method to solve three robotic manipulation tasks, outperforming the chosen baselines in terms of task success rate, safety, imitation error and sample efficiency. ", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n- The paper develops a method for learning geometric fabrics from demonstrations and demonstrates the capabilities of the resulting policy. \n- The results are compared to those of many baselines in well-chosen experiments and show superior performance in all the defined metrics.\n- code is provided in the supplementary material\n\nWeaknesses:\nThere remain some unclarities, as described in the issues section.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I recommend accepting the paper, since it proposes a promising approach for learning geometric fabrics from demonstrations. The capabilities of the method are demonstrated in well-chosen experiments and its superior performance w.r.t. many baselines is shown. \n", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "2: Unsure, difficult to assess.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {"issues": "- I assume the training is done in an end-to-end manner, with the loss in eq. (9) in appendix D defined in the root space. But this is not stated clearly anywhere. I also think it would be beneficial to have more on the training in the main paper and not to defer it completely to the appendix, since it is an integral part for understanding the paper. \n- In line 241: I assume the '20 arbitrary object poses' used for evaluation where never seen during training, correct? I am unsure due to the ending of the sentence '... used for demonstrations.', which might refer to the 20 poses or to the table.\n- In line 243: Why do you use the mean of the target poses seen in training? Why not use an arbitrary target pose just as it is done for the object poses? Also, do I assume correctly that the target pose is the same for all 20 evaluation runs?\n- concerning RMPs: Are the metrics learned? If not, how are they chosen?\n"}}, {"review_id": "mZbj-vUKJVJH", "reviewer": "Reviewer_1QRC", "summary": "This paper proposes to learn the parameters of policies encoded via geometric fabrics from demonstrations. Specifically, the paper focuses on learning the policy parameters for a hand-and-arm system, whose motion is generated by two geometric fabrics describing the nominal behavior of the system in task and joint space and a geometric fabric inducing an additional potential function. The difference geometric fabrics are then combined following the principles of RMPflow, where the Riemannian motion policies are replaced by the geometric fabrics. The paper then proposes to encode the geometric fabrics' parameters as neural networks, and to learn them from demonstrations. The paper tests the proposed approach in 3 different manipulation tasks, and assesses its performance with respect to several neural-network-based policies and to a Riemannian motion policy.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n\n- The proposed approach is interesting as it allows the learning of the parameters of a geometric-fabric-based policy from demonstrations. This may be beneficial as geometric fabrics may be cumbersome to design by hand. \n- The proposed approach is nicely illustrated with 3 real-world experiments involving a complex robotic system (multi-fingered hand mounted on a robotic arm). The paper also made an nice effort in comparing their approach with respect to other state-of-the-art policies on the real system. \n\n# Weaknesses\n- The novelty of the paper is rather limited: It mostly follows the ideas presented in [11] (Rana et al, CoRL'20), where the parameters of Riemannian motion policies were encoded as neural networks and learned from demonstrations. As geometric fabrics generalize Riemannian motion policies [12], the contributions of the paper are rather incremental.\n- The presented geometric-fabric-based policy is very specific to the system and tasks at hand. Although the presented approach may also be used for geometric-fabric-based policy specified for other systems and tasks, the paper does not mention it. Consequently, it is also unclear how the proposed approach may perform for and scale to different systems/tasks. \n- For example, would the proposed approach scale to tasks considering more than 3 geometric fabrics? This is unclear to me, as this would result to an increase of the number of parameters to learn, which may be unfeasible without a very large number of demonstrations (considering the large number of demonstrations used for the tasks considered in the paper, see next comment).\n- 80 demonstrations are used for the two first tasks evaluated in the paper. Learning-from-demonstration (LfD) approaches usually use a handful of trials, and aim at facilitating the learning of skills from few demonstrations (including generalizable skills). In that sense, providing 80 demonstrations to essentially learn a picking and a pushing task seem very cumbersome and not adapted for a viable LfD approach. This is even more striking when considering that only 6 demonstrations were provided for the third task considering in the paper. \n- Following the previous remark, the analysis on sample efficiency conducted in the first task tend to show that a very large number of demonstrations is needed to obtain a decent performance on the first task. Although the proposed approach is the most sample-efficient according to this analysis, the required number of demonstrations remain considerable. In my opinion, this is a clear limitation, especially when considering that the demonstrations were acquired from a human (via teleoperation).\n- The paper claims to learn high-dimensional policies. Although the hand-and-arm system originally has 23 DoFs, the dimensionality of the geometric fabric describing the nominal behavior of the system in task space and of the geometric fabric encoding the potential function is reduced to 8 by considering the position of the end-effector (3 dimensions) and the eigenspace of the hand (5 dimension). These two low-dimensional geometric fabrics clearly encode the most important part of the motion, as the remaining geometric fabric serves as a residual policy, filling in the nullspace of the 8-dimensional space. Therefore, I find the claim of learning high-dimensional policies somewhat exaggerated, as a lower-dimensional space built on prior knowledge is considered for the most important part of the motion.\n- Additional details should be given in order to make the paper self-contained. For example, as learning the geometric-fabric-based policies from demonstrations is the main contribution of the paper, the considered loss should appear in the main text. Similarly, it would be beneficial to define the imitation error in the main text. The input of all networks are not detailed in the main paper, while features and states are mentioned as inputs in the Appendix. Additional explanations on this would be appreciated. Which features and states are considered here? Also, the roles of $[M_*, \\pi_*]$ (section 4.2) and the decomposition of the terms $\\pi_k$ (Figure 2) are not explained. \n- The variable $\\pi$ is used to describe the geometries (Section 3, 4) and the policy (Section 5), which may be confusing for the reader.\n- In Section 2, in the \"exploiting structure\" paragraph: probabilistic movement primitives ([33], [34]) are not dynamical-system-based approaches. In general, the paper tends to cite long lists of papers without explaining each of them in details (e.g., [1-9], [29-39]). Selectively citing and explaining some of these papers may be more interesting when describing the related work.\n- As the paper is concerned with learning from demonstrations and does not consider reinforcement learning, the background section on reinforcement learning does not seem necessary.\n\n## Open questions\n- How many parameters are learned for the final policy?\n- The low performance of Riemannian motion policies compared to geometric fabrics (and other approaches) in the presented experiments was surprising to me, as geometric fabrics are essentially an improvement on Riemannian motion policies. Is there an explanation for this?\n\n## Cosmetics\n- Consider displaying the y-axis from 0 to 100% for all success rate graphs.\n- The labels in the figures are generally too small.\n- Please improve the format of the references. Many papers cited as arXiv preprints were actually published in conferences and journal. Double check upper cases (e.g., [14] hamiltonian -> Hamiltonian). Some journals are abbreviated, some are not (e.g., [23], [27] vs others). Several citations display twice the year and/or the publisher (e.g., [13], [29], [48]). [10] and [46] cite the same paper. Some papers with many authors are cited usng \"et al\", some not ([33], [12] vs [7])", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Overall, the proposed approach is interesting and may be valuable when deploying geometric fabrics on real robots. However, the contributions of the paper are rather incremental and mostly result from a combination of the ideas in [11] and [12]. Moreover, the presented architecture is very specific to the considered hand-and-arm system (especially concerning the design of the underlying geometric fabrics), and it is unclear how it may scale to other systems and tasks. A very large number of demonstrations was required for 2 out of the 3 considered tasks, thus hindering the general applicability of the proposed approach. Finally, several clarifications are required to make the paper self-contained.\n\n\n--------\n**Post-rebuttal comments**\n\nI would like to thank the authors for their detailed answer to my questions and comments. I appreciate the authors' investment in the rebuttal. \n\nHowever, I think that the main issues of the paper remain after the rebuttal phase. The contributions as initially stated in the paper remain incremental, while I find that the additional contributions stated during the rebuttal are vague and barely reflected in the paper. Moreover, the issues about data efficiency have not been completely answered. Finally, my doubts about the effective dimensionality of the NGF remain as there is still some confusion about the role of the eigenspace for controlling the 16 DoFs of the hand. Although I agree that each eigengrasp controls the 16 DoFs of the hand, the benefit of using an eigenspace is that the final posture of the hand can be expressed as a function of the amplitudes along each of the eigengrasp direction. As 5 eigengrasps are used here, the posture of the hand should therefore be expressed as a function of the 5-dimensional amplitude vector (and not via the 16 original DoFs of the hand), which reduces the dimensionality of the problem.\nTherefore, I maintain my initial evaluation.", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are not well addressed", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {"issues": "As mentioned in the \"strengths and weaknesses\" section, the following points should be clarified or improved in the revised manuscript:\n- Novelty of the proposed approach;\n- Applicability of the proposed approach to different systems;\n- Very large number of demonstrations required to learn a task;\n- Effective low dimensionality of the main geometric fabrics vs claim of learning high-dimensional policies;\n- Clarifications on the content of the main paper;\n- References and figures formatting."}}, {"review_id": "c9UM3TAdll6", "reviewer": "Reviewer_WsPo", "summary": "This paper proposes to leverage learning-from-demonstration to learn the parameters of a Geometric-Fabric motion policy, parametrized by neural networks. The main motivation of this work is to leverage the structure provided by geometric fabrics in robot motion generation to address the problem of designing dexterous manipulation skills in high-DoF robotic systems. The method is tested on three different manipulation tasks that involving picking, pushing and extraction skills. ", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n1. Real experiments on hardware:\nOne of the key aspects of this paper is the real experiments reported in Section 6 and shown in the paper. On the one hand, it is very clear how introducing the structure of geometric fabrics brings benefits when learning dexterous manipulation skills, which is very clearly observed in Fig. 3. On the other hand, the information provided in Appendix and the available code may provide the opportunity to reproduce the results reported in the paper and/or employ the proposed method in other works. \n\n2. Facilitate the design of geometric fabrics via LfD:\nIt is definitely important to highlight that the design of the parameters of geometric-fabric policies may be tedious, and leveraging LfD to address this problem brings some benefits when using this robot motion generator in practical settings. \n\n\n**Weaknesses** \n1. Unclear contributions:\n   * The paper clearly states three contributions in lines 59-65, claiming novelty on the use of LfD for geometric-fabrics policies, thorough analysis of the policy structure, and the first framework to train and generate joint-space motions on high-DoF dexterous manipulation using learning methods. Below I explain the reasons why I disagree with some of these claims:\n      - Using LfD and Geometric fabrics seems to be relatively new. However, the combination of LfD to learn RMP parameters (a very similar approach, as acknowledged in the paper) was already proposed by Rana et al [11]. There is nothing particularly new in the design of the networks or learning process that is specially designed for Geometric Fabrics. In this sense, it is hard to see this as a novelty. Going beyond this, Geometric fabrics is an approach proposed in [12] and [13], the tree-like structure to combine different policies was proposed in the RMPflow work [45] and also improved in [10]. \n\n     - Analysis of the importance of the policy structure is somehow confusing. On the one hand, I agree that comparing agains FF and LSTM networks is a good way to show that policies with some meaningful structure provide benefits when it comes to learn motion policies. However, the comparison against RMP, which is also an approach that provides a strong policy structure is not analyzed in detail. It is unclear why such big differences exist given the fact that RMPs may also be learned from demonstrations and have shown successful experiments in past works. The paper should provide clearer evidence or a deep discussion on the possible reasons of the observed performance differences.\n\n2. Related work:\n\n   Although the paper does a good job by reviewing different works using IL and RL to learn dexterous manipulation skills, there are some points I think deserve special attention:\n\n   - Two recent works on learning dexterous manipulation (on high-DoF robots) from demonstration are missing [a,b] (which I think are very close, in practical terms, to the concepts and experiments reported in this paper):\n      [a] A. Gupta, C. Eppner, S. Levine and P. Abbeel, \"Learning dexterous manipulation for a soft robotic hand from human demonstrations,\" 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016\n      [b] P. Ruppel and J. Zhang, \"Learning Object Manipulation with Dexterous Hand-Arm Systems from Human Demonstration,\" 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020.\n\n   - In the review of IL approach, the paper claims that one of the weaknesses of the related works is the data efficiency, yet the proposed approach in this paper was tested with 80 demonstrations for one of the experiments, which does not seem any different from other IL approaches. \n\n   - In lines 102-104, the paper claims that NFG provides a more “appropriate inductive bias” than RMPs. But it is unclear what the paper means with “more appropriate”. This claim needs to be clarified. \n\n   - In more general terms, the paragraph on exploiting structures does not provide any insights on the differences between the structure that NFGs bring when compare to more classical motion primitives such as DMP and ProMPs, which are cited but not discussed. \n\n3. Experiments:\nAs pointed out above, the comparison against FF and LSTM networks clearly show the advantages of having structure in the robot policy representation, this is a great result to report. However, I would like to suggest two important improvements in the experiments:\n   - Figure 3 and 4 are all about showing the benefits of including structure for task performance and data efficiency. However, if there is an approach in the literature that stands out as a method that is data efficient and includes structure is DMPs, cited in this paper. DMPs provide a second-order DS structure, can be learned from a single demonstration (in some mild cases) and are mainly learned using LfD. So, this seems to be a very good baseline to use to compare against to. Either if the DMP is learned in task or joint space, this may provide some additional insights on the performance of NFG against other approaches that provide strong structure. \n    - It is unclear if RMPs parameters are learned as proposed by Rana et al, or if this paper considers manually-designed RMPs for comparison purposes.\n    - Figure 4, in my opinion, shows that NFG are **not** data efficient. They seem to be better than FF and LSTM, but considering success, in terms data efficiency, when using 60 human demonstrations is not data efficient in LfD literature (consider DMP or ProMPs, which can be trained with a handful of demonstrations). More importantly, real applications of LfD approaches that demand the human teacher to provide more than 10-15 demonstrations are not practical. This should be considered a limitation of the approach and discussed in the corresponding section. \n\n*Lack of details in proposed approach*\n- Clarify why the metric-weighted projection matrix $\\mathbf{P}_e$ boils down to an identity-matrix metric in line 193. \n- Line 217 refers to the appendix to understand the policy optimization and loss function, but in my opinion, both are core components of the proposed approach and should appear in the main text. \n\n     ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper leverages LfD to learn the parameters of Geometric-Fabric policies. In this regard, the paper does not clearly state the main differences w.r.t all the previous works on RMPs and Geometric-Fabrics, as some of them (in particular Rana’s work) highly overlap on the ideas and concepts that this paper builds on. Along a similar line, due to the high overlap with previous works, it seems that several claims about novelty are hard to see. Last but not least, the experiments fall short on providing strong evidence on data efficiency, and there is a lack of comparison against more classical movement primitives, that theoretically speaking, may also provide strong structure and data efficiency, as claimed for NFGs.\n\n------\n\n**Post-rebuttal recommendation**\n\nThank you very much for the rebuttal and interesting discussion. I appreciate very much the time and effort invested during the rebuttal phase. Below you can find my thoughts on my final recommendation:\n\n1. As I stated in the rebuttal, GFs and RMPs are very similar geometric control approaches, with the former being a sort of generalization of the latter. As such, nearly every extension made about RMPs (RMPflow, RMP2, RMP learning via demonstrations) can be applied to GFs. In fact, the paper leverages RMPflow computations and similarly, the proposed extension to learn the GF parameters from demonstrations is very close to the RMP extension of Rana et al.,(2019). In this regard, I find the authors' reply unsatisfying.\n\n2. Another aspect that is not fully addressed is sample efficiency. In my experience, well-known LfD approaches are easily trained with 10-15 demonstrations (e.g., DMPs, GMMs, CNPs, ProMPs, to mention a few) for fairly complicated tasks. Some of these approaches, extended to complex settings like Riemannian manifolds, have been trained with such small datasets. So, in this regard, 60+ demonstrations do not align with sample efficiency claims. Thus, the authors' rebuttal was unsatisfying.\n\n3. I fully agree with the authors' on that GFs can be seen as inductive bias in robot motion skill learning (as discussed in my rebuttal). However, this is not specific to this paper, but it is a contribution of the GFs papers themselves. On a similar line, if such inductive bias were a game changer when learning this kind of policies, the paper falls short on providing evidence on this regard: RMPs, which also bring inductive bias to the learning procedure, seem to perform similarly w.r.t NN or LSTM (in some cases). Then it is unclear why the inductive bias that RMPs bring to the problem do not provide improvements over these baselines and why GFs do instead.\n\n4. Last but not least. I think the work reported in this paper is very interesting, it has good merits, as I pointed out in my review. But I would like to also second the feedback of reviewer 1QRC, which highlights other aspects that I overlooked in the paper and have not been properly addressed.\n\nThen, I keep my recommendation. ", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {"issues": "The main issues of this paper are: unclear contributions (it is unclear how much novelty the paper brings when compared to all the RMP and Geometric Fabrics literature), confusing claims on related works and missing references, and the provided experiments. Please see my comments above for suggestions. "}}, {"review_id": "0ejgC05Xhu", "reviewer": "Reviewer_zkro", "summary": "The authors propose a new learning from demonstration architecture that creates a structured policy capable of controlling robots with high DoF from a limited number of demonstrations. The paper presents the architecture and learnable parameters of Neural Geometric Fabrics (NGF) by first introducing the relevant prior work on Geometric Fabrics. An extensive comparison of NGF and several baselines is presented demonstrating the success rate and safe deployment of NGF on three tasks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n* The paper is well written and shows a clear progression from the relevant prior work to the proposed architecture. The authors make a good effort to simplify the complexity of the method for the reader.\n* The authors select 4 baselines for comparison with NGF and provide a comprehensive evaluation of the results. \n* The supplementary material includes a video verifying the robot deployments and further evaluations of the baselines and NGF method along with architecture parameters and algorithms.\n\n# Weaknesses\nThere are no large or obvious weaknesses to this paper. However, another baseline for comparison that induces structure for learning motion trajectories would be helpful for the reader to validate the improvements NGFs make to the learning from demonstration area. The only method that comes to mind is \"Conditional Neural Motion Primitives\" although there are likely better ones. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "This paper introduces a novel, structured method for learning from demonstration and proves its sample efficiency and ability to control robots with high degrees of freedom. The inclusion of multiple baselines and a thorough experimental section make this a considerable contribution. ", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "issues": "* In Figure 2, the numbering in the caption does not easily follow the graphic. For readability could numbering or steps be added to the graphic. ", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)