Venue: CoRL 2021 Conference

Reviews:

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: This paper generalizes "Probabilistic Movement Primitives" from Euclidean spaces to Riemannian manifold.
The basic concepts in PMP (learning from demonstration, trajectory modulation, blending, task parametrization) 
are mapped to the corresponding concept on the manifold. The mapping is done via geodesic regression, utilizing  
the properties of smooth manifold. the method is demonstrated both on a synthetic data, and on a physical platform.
the simulations confirm the main claims of the paper. 
strengths_and_weaknesses: Strengths:
- the paper is well written, provides the required background
- the analogy between the PMP and the proposed method and the advantage is clear.  
- the experiments are convincing. 
- the videos in the supplementary are helpful and insightful. 
- the limitations are reasonable. they are discussed, and the future steps are proposed.

Weakness:
- it would be worth to elaborate on the complexity of different complement of the method. 
- adding more diverse experiment on the physical platform will be useful

summary_of_recommendation: I like this paper, because it has all the required: motivation, problem definition, rigour solution, validation. 




issues: --
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: This paper presents a principled extension of "probabilistic movement
primitives" or ProMPs from linear Gaussian models to data defined on
Riemannian manifolds. This is important for learning trajectories, since often
the trajectory of interest is specified in terms of orientations and poses,
which can be elegantly represented in terms of Riemannian manifolds. Consequently, the
extension of ProMPs to Riemannian manifolds is well-motivated, but involves a
nontrivial application of some fairly advanced machinery from Riemannian
geometry. The resulting approach is demonstrated on synthetic data and using a
real robot.
strengths_and_weaknesses: ## Strengths

Overall, I felt this was an excellent paper.

- The paper is well-written and organized. 

- The contribution is clear and relevant; the extension of probabilistic
movement primitives to Riemannian manifolds is well-motivated.

- The formulation is concise and easy to follow (not always easy to achieve when
  explaining Riemannian geometry)
  
- The method is demonstrated on synthetic data (which were really helpful for
  "visualizing" what was going on) and with real robot experiments.
  
- The animations in the supplemental video (e.g. adding via points) were very
  well done.

## Weaknesses

I have only minor comments for possible improvement of the paper, see Issues
below.

- The only "weakness" that stands out is the demonstration of only a single
skill with the real robot. It would be really neat to show more trajectories. As
mentioned below, though, the magnitude of the "theoretical" contribution makes
this a bit minor.

- I'd also suggest discussing a bit more about the timing / computational
efficiency. I'm particularly interested in the second issue raised in the
Discussion (regarding geodesic regression). How much more costly (e.g. in units
of time or relative order of magnitude) is this vs. the corresponding linear
system for ProMP?
summary_of_recommendation: I am recommending strong acceptance. This is a really nice paper. The proposed
work is well-motivated and clearly explained. Beyond that, results with
synthetic data and real robot experiments are provided. My only comments on
possible areas for improvement are minor.
issues: ## "Orientation" ProMPs

- The paper refers to "Orientation" ProMPs, but I don't see anything in the
  derivation in Section 3 that restricts this particular approach to
  orientations specifically - rather, this seems to be a general extension of
  ProMPs to arbitrary Riemannian manifolds (though admittedly, I don't have an
  immediate application to trajectory learning employing other manifolds that
  comes to mind). Might be worth a mention (maybe the authors have thoughts on
  this?)

## Discussion

- In the discussion, the issue of computational cost is raised, but I didn't see
  any quantitative mention about this (e.g. is there a noticeable increase in
  computation time). It's possible I missed something, but clarification here
  would be helpful.

## Minor Comments

- I recommend the excellent reference [1] for more detail on manifolds,
  especially for the concept of parallel transport. This might help a reader
  needing help with these concepts (it is a good supplement to the Absil,
  Mahony, and Sepulchre reference).

- Maybe more results could be provided (e.g. learning other skills than
  "re-orient"), but given the theoretical contribution this is minor.

[1] Boumal, Nicolas. "An introduction to optimization on smooth manifolds."
 Available online, Aug (2020). http://sma.epfl.ch/~nboumal/book/index.html
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper presents Riemannian formulation of Probabilistic Movement Primitive (ProMP) that enables the representation of quaternion trajectories which adheres to Riemannian geometry. In doing so, this paper extends ProMP which previously can only represent trajectories with Euclidean geometry, to those with Riemannian geometry, unlocking the possibility of encoding the full 6D pose trajectories and its uncertainty/probabilistic representation. The paper also explains how to perform Learning from Demonstration, trajectory modulation, blending, and task parametrization with this Riemannian ProMP.
strengths_and_weaknesses: Strengths:
The paper explains the background on ProMP, Riemannian Manifold (tangent spaces, geodesic, exponential and logarithmic maps, and parallel transport), and geodesic regression very well.

Weaknesses:
The explanation of Orientation ProMPs is sometimes unclear, or difficult to grasp/follow; sometimes I lose track during some algebraic manipulations.

Moreover, the Log() and Exp() mapping in the supplementary materials are different from those in the referenced paper: A. Ude, B. Nemec, T. Petric, and J. Morimoto. Orientation in Cartesian space dynamic movement primitives.
No clarification is provided yet.

Furthermore, naturally the dimension of the tangent space of 3D orientations (regardless of representation: quaternion, axis-times-angle, or rotation matrices) are 3; which makes sense because this tangent space is the same space as the space of angular velocities (omega vector in R^3). However, it is stated in line 205 that wn in tangent space TpM is a subset of R^4. No clarification is provided yet.

See `Issues` for suggestions of improvements.
summary_of_recommendation: The paper contains novelty and deserves a publication, but can be improved by following the recommendations in the `Strengths and Weaknesses` or `Issues`.
issues: 1. The explanation of Orientation ProMPs is sometimes unclear, or difficult to grasp/follow; sometimes I lose track during some algebraic manipulations. For example, on line 167, it would be helpful to more explicitly state what are the dimensions of the matrices U, xi, Xi, and u. Also, it would be very helpful (in maximizing the impact of the paper) to upload the source code implementation of the paper into the supplementary materials.

2. Moreover, the Log() and Exp() mapping in the supplementary materials are different from those in a referenced paper A. Ude, B. Nemec, T. Petric, and J. Morimoto: Orientation in Cartesian space dynamic movement primitives, as well as in a non-referenced paper J. Solà, J. Deray, and D. Atchuthan: A micro Lie theory for state estimation in robotics (https://arxiv.org/pdf/1812.01537.pdf).
Maybe it's useful to comment on what caused the differences.

3. Furthermore, naturally the dimension of the tangent space of 3D orientations (regardless of representation: quaternion, axis-times-angle, or rotation matrices) are 3; which makes sense because this tangent space is the same space as the space of angular velocities (omega vector in R^3). Can the author comment on why it is stated in line 205 that wn in tangent space TpM is a subset of R^4?
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper presents an extension of the probabilistic movements primitives framework for Riemannian Manifolds.
The authors propose a full ProMP framework adapted to the Riemannian Manifolds, which includes imitation learning, conditioning, blending, task parametrization.

strengths_and_weaknesses: The theory is well presented and well exposed. The formulation seems solid and quite easy to use.
However, the contribution is really incremental, as it consists mainly of combining existing techniques to enhance the ProMP framework.
The experimental evaluation is good, but a bit too simplistic, and it's not showing in a sufficiently convincing manner the advantage of considering the quaternions instead of euclidean variables to control robotic manipulator. While the authors prove that their method is better than plain ones, I would like to see more complex robotics experiments where orientation is fundamental e.g. peg-in-a-hole or other similar robotics tasks.
summary_of_recommendation: I think this is a borderline paper. The theoretical framework is solid, and the solved problem is really interesting. However the contribution is a bit incremental and the experimental evaluation, particularly on real robots is quite weak.
If I had the possibility, I would vote for borderline (nor weak accept, nor weak reject). 
However, as I think that this paper could be improved, also in the rebuttal, I selected weak reject.
issues: - Add a more complex robotic experiment to justify your claims and to show the capabilities of the robot.
review_title: null

Meta-review: The reviewers agree that this is a very well written and thought through paper.
Most minor concerns regard some additional clarifications that should be included in the revised version.
The only major complaint is about the complexity of the provided experiments.
If possible some additional more complex real robot experiments would benefit the paper and make a stronger case for the approach.

## Post-Rebuttal Update:
The authors successfully addressed and clarified most of the raised issues and further improved the paper through the rebuttal phase.
Even if I would agree that the work is incremental, this would not a be a major concerns as long as the work is not trivial, which is not the case here.
I would ask the authors, however, to reference and discuss

[Gaussians on Riemannian Manifolds: Applications for Robot Learning and Adaptive Control](http://ras.papercept.net/images/temp/IROS/files/3131.pdf)  by Sylvain Calinon

in the camera ready version.
Decision: Accept (Poster)