Learning Riemannian Stable Dynamical Systems via Diffeomorphisms
Authors: Jiechao Zhang
Hadi Beik Mohammadi
Leonel Rozo
Venue: CoRL 2022 Poster (2022)
Keywords: Dynamical systems
Riemannian manifolds
Motion learning
Learning from demonstrations
Neural ODEs

Abstract: Dexterous and autonomous robots should be capable of executing elaborated dynamical motions skillfully. Learning techniques may be leveraged to build models of such dynamic skills. To accomplish this, the learning model needs to encode a stable vector field that resembles the desired motion dynamics. This is challenging as the robot state does not evolve on a Euclidean space, and therefore the stability guarantees and vector field encoding need to account for the geometry arising from, for example, the orientation representation. To tackle this problem, we propose learning Riemannian stable dynamical systems (RSDS) from demonstrations, allowing us to account for different geometric constraints resulting from the dynamical system state representation. Our approach provides Lyapunov-stability guarantees on Riemannian manifolds that are enforced on the desired motion dynamics via diffeomorphisms built on neural manifold ODEs. We show that our Riemannian approach makes it possible to learn stable dynamical systems displaying complicated vector fields on both illustrative examples and real-world manipulation tasks, where Euclidean approximations fail.


Reviews:
[{"review_id": "VK2LC8IJYh", "reviewer": "Reviewer_SPAS", "summary": "The paper presents a method to learn robot motions that are restricted to specific Riemannian manifolds using stable dynamical systems and diffeomorphisms. The proposed technique combines recent developments from neural dynamics and geometry-aware formulations of robot motions in order to ensure both stability as well as geometric constraints on trajectories (e.g., trajectories belong to a specific Riemannian manifold). Experiments demonstrate that proposed method is better at respecting geometric constraints of the trajectories, compared to a recent method capable of learning stable dynamical systems via diffeomorphisms. ", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\n- The proposed methodology is well explained and all necessary details to understand the core ideas are provided. Further, I appreciated the additional comprehensive background and details in the complementary material.\n- The related work is section does a pretty good job of discussing some of the most relevant methods for dynamical-systems based LfD, and positions the paper well within this subfield.\n- The proposed method is theoretically sound and brings together a collection of recent ideas from different related communities in machine learning, dynamical systems, and robotics.\n- The experiments conclusively demonstrate that proposed method can learn stable dynamical systems on a Riemannian manifold and that the reproductions obey the inherent geometric constraint.\n\n### Weakness\n\n- While the paper does a fantastic job and detailing the theoretical differences between the proposed approach and that of existing approaches, I felt that the expressed motivation for robotics was somewhat unsatisfactory. Indeed, examples of manifolds that are relevant to robotics are suggested (e.g., $SO(3)$ and $SE(3)$), the practical benefits of the claimed theoretical properties remains elusive. \n\n- On a related note: While benefits over existing methods that do not obey geometric constrains is clear (and demonstrated), benefits over methods that rely on Lie groups (e.g., Ref [10]) is somewhat unclear. Indeed, it is stated that not all manifolds are easily expressed as Lie groups. But this argument demonstrated in experiments. In fact, only one of the related approach was used as a baseline in the experiments (that too only in the simulated dataset).\n\n- The robot experiment involved relatively simplistic tasks that are unlikely to demonstrate the benefits of the nuanced theoretical differences between the proposed methods and the existing methods.  This problem is exacerbated by the fact that the physical robot experiments did not involve any comparisons with existing approaches and merely serves as a proof-of-concept of the approach. I find it hand to imagine how existing methods will fail in these tasks.\n\n- It might be helpful to slightly expand the discussion of related work to include a class of methods known as Riemannian Motion Primitives (RMPs) (e.g., [1]]) and Geometric Fabrics [3]. I do not believe that these methods subsume the contributions of the proposed method but these methods employ very similar structures in the policy and it would be worthwhile to understand how the proposed approach differs from them.  \n\n- The technical advances over Ref [21] (the Manifold ODE paper) could be made more explicit so that the reader will understand why one can not view the proposed method as a trivial application of the technique in Ref. [21] to robotics tasks. \n\n### References\n\n1.  Rana, M. A., Li, A., Ravichandar, H., Mukadam, M., Chernova, S., Fox, D., ... & Ratliff, N. (2020, May). Learning reactive motion policies in multiple task spaces from human demonstrations. In Conference on Robot Learning (pp. 1457-1468). PMLR.\n2. Van Wyk, K., Xie, M., Li, A., Rana, M. A., Babich, B., Peele, B., ... & Ratliff, N. D. (2022). Geometric fabrics: Generalizing classical mechanics to capture the physics of behavior. IEEE Robotics and Automation Letters, 7(2), 3202-3209.\n ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper is well written and provides an elegant theoretical approach to learn stable robot motions in complex spaces. The theoretical presentation and exposition is commendable and the experiments serve to fortify some of the claims made in the paper. My only criticism stems from the fact that the practical benefits of the nuanced theoretical are either i) mentioned but not substantiated in the experiments, or ii) not mentioned at all.", "technical_quality": "Excellent: Very convincing and thorough.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "issues": "Please go over my comments from above and provide clarifications where appropriate, and push back on points that reflect my misunderstanding.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {}}, {"review_id": "L5-uR0eCWTF", "reviewer": "Reviewer_hQF2", "summary": "The paper considers the problem of learning stable dynamical systems on manifolds. The main approach proposed in the paper combines existing methods in 1) learning stable dynamical system via diffeomorphisms and 2) normalizing flows on manifolds. The paper additionally proposes solution for computing the pullback operator. The proposed method is validated on a simulated task and 2 real robot tasks. ", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n- **Clarity.** Despite being technically involved, the paper is well-written. The authors provided background on differentiable geometry in a concise and easy-to-follow way. Readers with little background on differential geometry should be able to follow along.\n- **Organization.** The paper is well-organized. The author provided concise and useful differentiable geometry background and normalizing flow. The technical section provides detailed description of the proposed method. The equations are well-explained. \n- **Experiments.** The authors showed significant improvement over the baseline algorithm on a simulated task. The paper also contains a real robot experiment, showing that the proposed method is scalable to real hardware. \n### Weaknesses\n- **Global asymptotic stability.** The proposed method is claimed to \"ensure global asymptotic stability\", which is the main motivation behind the diffeomorphism parameterization. It seems, however, that *global* asymptotic stability, at least for a single equilibrium point, may not be possible. This is due to the Poincaré–Hopf theorem: the sum of the Poincaré indices over all the isolated zeroes of a vector field must be equal to the Euler characteristic of the manifold. Globally stable local equilibrium point means a single zero with index 1. However, for, e.g., 2-sphere which has Euler characteristic 2, there must exist at least another zero. This means that globally asymptotically stable is not achievable for such a manifold. I might be wrong here so maybe the authors can elaborate more on the existence of globally asymptotic equilibrium points.\n- **Usage of CNF.** In the conclusion, the authors mentioned computational limitation caused by the Neural MODE. Can the authors provide more explanation on why they still choose this diffeomorphism parameterization rather than others in the normalizing flow literature? Are other parameterizations infeasible for learning diffeomorphisms on manifolds?\n- **Comparison with baseline.** It would be nice if the author can also compare their proposed method to the baseline in the real robot experiment. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper considers the problem of learning stable dynamical systems on manifolds. The main approach proposed in the paper combines existing methods in 1) learning stable dynamical system via diffeomorphisms and 2) normalizing flows on manifolds. The paper additionally proposes solution for computing the pullback operator. \n\nThe paper is well-written. The authors provided background on differentiable geometry in a concise and easy-to-follow way. Readers with little background on differential geometry should be able to follow along. The authors showed significant improvement over the baseline algorithm on a simulated task. The paper also contains a real robot experiment, showing that the proposed method is scalable to real hardware. \n\nHowever, there is some concern on the theoretical soundness of the paper. I am happy to increase my score if my concern is addressed. ", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {"issues": "(The list below is the same as **Weaknesses**)\n- **Global asymptotic stability.** The proposed method is claimed to \"ensure global asymptotic stability\", which is the main motivation behind the diffeomorphism parameterization. It seems, however, that *global* asymptotic stability, at least for a single equilibrium point, may not be possible. This is due to the Poincaré–Hopf theorem: the sum of the Poincaré indices over all the isolated zeroes of a vector field must be equal to the Euler characteristic of the manifold. Globally stable local equilibrium point means a single zero with index 1. However, for, e.g., 2-sphere which has Euler characteristic 2, there must exist at least another zero. This means that globally asymptotically stable is not achievable for such a manifold. I might be wrong here so maybe the authors can elaborate more on the existence of globally asymptotic equilibrium points.\n- **Usage of CNF.** In the conclusion, the authors mentioned computational limitation caused by the Neural MODE. Can the authors provide more explanation on why they still choose this diffeomorphism parameterization rather than others in the normalizing flow literature? Are other parameterizations infeasible for learning diffeomorphisms on manifolds?\n- **Comparison with baseline.** It would be nice if the author can also compare their proposed method to the baseline in the real robot experiment. "}}, {"review_id": "C6rl9tJEpdOz", "reviewer": "Reviewer_eXxr", "summary": "This paper builds on previous methods of learning stable dynamical systems for imitation learning, by starting with a stable vector field, and morphing it according to demonstrations, while remaining stable. Previous methods, such as Euclideanizing flows, rely on diffeomorphisms map to isomorphisms of Euclidean space. This paper defines invertible functions which operate on a wider class of Riemannian manifolds by using the differentiable integrator from Manifold Neural ODEs (MNODEs), to enforce the vector field on a specified manifold. The paper empirically evaluates the usage of the diffeomorphism created by the MNODE on the LASA dataset on S^2 and two real robot environments.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The idea is sound, and the execution is good. Methods to use diffeomorphisms to maintain stability has been estabilished in (Rana et al, L4DC, 2020; Urain et al., IROS, 2020; Zhi et al, L4DC, 2022), and the proposed extension via Manifold Neural ODEs (MNODEs) is quite logical. The execution is also good, with real robot experiments.\n\nI'm less enthusiastic with the motivation here. I've dabbled with Manifold Neural ODEs, and they are much slower than differentiable integrators from the torchdiffeq library. I'm not sure when there would be a setup where the fairly minor performance gains are worth the additional time expenditure. Regardless, I think it is useful to extend diffeomorphism-based imitation learning to restricting it to a wider range of Riemannian manifolds, and I vote for acceptance.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The method is sound, and the experiments are the correct ones. More experiments and results on the LASA dataset could be included, the \"S\" is a classic, and there are quite a lot of characters -- why only select such a small number of letters? The results presented are graphical, it would be good to see some numbers over more characters. \n\nThe limitation sections make note that NMODEs are slow. Runtime is quite important here -- if it is sufficiently fast, perturbance can be accounted for online. Please quantify and report on it.\n\nAdditionally, Zhi et al. \"Diffeomorphic Transforms for Generalised Imitation Learning\", L4DC 2022 and Urain et al., \"ImitationFlow: Learning Deep Stable Stochastic Dynamic Systems by Normalizing Flows\" IROS 2020 are relevant references that should be included.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {"issues": "-> run experiments on the whole LASA dataset, not just 4 cherry-picked characters;\n-> report on runtime;\n-> further clarify (in writing) motivations where the manifold variant is useful -- the LASA on S^2 is slightly contrived;\n-> include Zhi et al. and Urain et al., which are very relevant."}}, {"review_id": "0NIAJs9h1-o", "reviewer": "Reviewer_AVAQ", "summary": "This work seeks to learn motion dynamics from human demonstrations. The work resembles Neumann et al., 2015, such that they view human demonstrations as trajectories from a stable dynamical system on a Riemannian manifold and enforce stability by mapping the dynamics into a latent space whose dynamics are manually-designed to be stable. As such, learning stable dynamics reduces to learning a diffeomorphism between the observed and latent manifolds. This paper extends the methods from Neumann et al., 2015, by encoding stable dynamics on a latent Riemannian manifold using geodesic vector fields, which they show to be Lyapunov stable. This forms one of their main contributions, along with a method for learning the diffeomorphism using Neural Manifold ODEs, which allows them to calculate the inverse diffeomorphism by simply reversing the integration time interval. They demonstrate their method on the LASA dataset projected onto the 2-sphere and on two real-world manipulator experiments.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n1. Mapping the system to a latent Riemannian manifold and encoding stability via a geodesic vector field is an elegant idea.\n1. Their approach to calculating the differential of the inverse diffeomorphism by reversing the integration time interval is neat.\n2. The real-world experiments verify that the method works in practice. I found the video in the supplementary material helpful for understanding the real-world experiments.\n   - Perhaps the authors could add a link to the video in the experiments section?\n\n**Weaknesses**\n\n1. The experiments have not verified whether or not the latent space needs to be a Riemannian manifold to ensure that the stable dynamical system temporally satisfies geometric constraints. Can the diffeomorphism not learn a representation that preserves these temporal geometric constraints?\n   - This paper assumes that the latent space should be a Riemannian manifold but more experiments are needed to support this claim. As a minimum, I would like to see RSDS compared to the original Euclideanizing Flows in Neumann et al., 2015. This is because the papers *modified* Euclideanizing Flows implementation maps the system to a latent Riemannian manifold (instead of Euclidean) and parameterises the diffeomorphism with a Neural ODE instead of a single-layer neural network with the layer resembling an approximated kernel machine. \n   - As such, it is not possible to infer why the *modified* EuclideanizingFlows implementation performed poorly.\n   - Comparing to results for EuclideanizingFlows with a diffeomorphism parameterised as a Neural Manifold ODE would allow direct comparison.\n2. The paper restricts itself to diffeomorphisms parameterised as Neural Manifold ODEs.\n   - Although it is perfectly fine to limit the scope of their work, I think that the authors could motivate the Neural Manifold ODEs better as it was not immediately clear to me why a Neural MODE was being used to learn the diffeomorphism.\n   - My current understanding (after several reads) is that the Neural Manifold ODE is useful for calculating the pullback operator because the inverse of the diffeomorphism can be calculated by reversing the time integral. Although the authors clearly state the problem of computing the inverse due to geometric constraints at the end of the first paragraph in Section 3.3, they do not state that they use Neural Manifold ODEs to do this. In my opinion, the authors should motivate the use of Neural Manifold ODEs and add some more sign posting. For example, by say something like this at the start of Section 3.2:\n     - \"Given Eq. 5 and a dataset of $N$ human demonstrations, the goal of learning stable dynamics is now reduced to learning $\\phi_{\\theta}(x)$ and $\\hat{k}\\_{\\gamma}(x)$. However, due to the geometric constraints arising from $\\mathcal{M}$, calculating the inverse diffeomorphism $D_{y}\\phi_{\\theta}(y)$ is problematic. For this reason, we leverage Neural Manifold ODEs to learn the diffeomorphism because we can calculate the inverse diffeomorphism by simply reversing the integration time interval, as we shall see in Section XX.\"\n3. Although not essential, the authors do not compare to experiments using the straightforward solution of calculating the pullback operator using the derivatives of the diffeomorphism.\n   + Does this method provide greater accuracy?\n\n**References**\nMuhammad Asif Rana, Anqi Li, Dieter Fox, Byron Boots, Fabio Ramos, Nathan Ratliff Proceedings of the 2nd Conference on Learning for Dynamics and Control, PMLR 120:630-639, 2020.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Overall, this paper does a nice job combining various methods leading to a novel approach for learning motion dynamics from demonstrations. They provide theoretical results in the appendix and they test their method in two experimental settings. They test their method's ability at reproducing trajectories on the LASA dataset projected onto $\\mathcal{S}^2$. That is, they test their method's ability to learn vector fields that encode letters projected onto a sphere. They then test its ability at reproducing trajectories from real demonstrations on a grasping task and a drawing task. The dynamic motion patterns required synchronised position-orientation trajectories and their results demonstrate that their method is capable of encoding this geometric structure. Although this paper extends the method from Euclideanizing Flows (Neumann et al., 2015), it presents a novel method for manually-designing stable dynamics in latent Riemannian spaces using geodesic vector fields and also provides a novel method for calculating the pullback operator by reversing the Neural Manifold ODE's integration time interval. The technical quality of this paper is also high. The authors detail key equations and intuitions in the paper and provide more rigorous theoretical contributions in the appendix.\n", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {"issues": "1. Incorrect statement about Euclideanizing Flows in Neumann et al., 2015 (citation 5 in paper) on line 39.\n    * Neumann et al., 2015 do not assume the data lies in the Euclidean space.\n    * On the contrary, they assume the data lies on a Riemannian manifold and use the diffeomorphism to map the system to a simpler Euclidean space where they can enforce stability.\n2. RSDS should be compared to the Euclideanizing Flows with a Neural Manifold ODE to enable the importance of the Riemannian latent space to be verified.\n3. I think the authors should be careful stating that they learn diffeomorphisms with normalising flows.\n    * In my eyes, this method is inspired (or resembles) normalising flows. This is because it is not concerned with normalising density functions. Instead, it is interested in mapping complicated human motion trajectories (on an observed Riemannian manifold) to geodesic trajectories (on a latent Riemannian manifold) by sequentially applying the change of variable theorem.\n    * Euclideanizing Flows received its name as it maps to a latent Euclidean space, just as Normalizing Flows received its name because the mapping seeks to normalise a latent density function. As such, the authors could consider calling their method something like \"RiemannianFlows\" or \"GeodesicFlows\".\n\nMinor fixes:\n- Line 20: there shouldn't be a comma before \"and\"\n  - Consider \"these movements makes hand coding an infeasible approach. Therefore, learning techniques...\"\n- Line 29: should \"model parameters optimization\" be \"model parameters' optimization\"\n- Line 129: the end of this sentence doesn't read well.\n- Line 199: \"a embedded\" should be \"an embedded\"\n- Line 244: should the n subscript in (x_{n,t}, \\dot{x}_{n,t}) be m?\n- Line 256: should \"vectors point inwards the sphere are\" be \"vectors point inwards towards the sphere's center, are\"\n- Sub-captions would make figure 2 easier to interpret\n- Figure 4 is referenced before Figure 3?\n- Figure 4 is hard to read.\n  + Consider making figure 4 only two figures wide with one figure"}}]

Author rebuttal:

Decision: Accept (poster)