Venue: CoRL 2023 Conference

Reviews:

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary_of_paper: In this work, the authors address the task of learning sets of valid configuration-space trajectories for arbitrary control tasks. The authors propose Motion Manifold Primitives, which is a mapping from a simpler low-dimensional space (i.e. R^m) to the configuration space. The core idea is that one can learn a density function over the latent space which can then be sampled to recover samples from the manifold in the configuration space. The authors’ second contribution is noticing that if the task parameter space has symmetries, and those symmetries can be cleanly and precisely described, and an equivariant group action can be defined, then one can formulate a mapping from the latent space to configuration space which is equivariant under all enumerated symmetries. The authors provide convincing empirical evidence that, in two scenarios with straightforward symmetries, their method demonstrates superior performance compared to other manifold-learning baselines.

review: # Originality

Manifold learning in and of itself is not novel - mapping a density function on a lower-dimensional manifold which is easy to sample from to a high-dimensional manifold is the backbone of most autoencoding methods. So learning a mapping between these manifolds for robotics tasks is not novel.

However, this work attempts to drill down into the structure of these manifolds in many real robotics tasks, and make some more precise guarantees on performance by exploiting this structure. Broadly speaking, this work is attempting to formally nail down some of the properties which are reasonably common in robotics tasks: 
Problem symmetries: the solutions set to motion planning problems, for instance, is often equivariant under some certain transformations of initial conditions. We can often enumerate these symmetries intuitively, but the community frequently struggles to deal with symmetries systematically (often offloading them to multimodal generative modeling frameworks).
Homeomorphism: If you change the task parameters slightly, you’ll probably be able to deform the solution manifolds smoothly as well. Intuitively, if you move an object back a little bit during a pick-and-place task, your manifold of pick-and-place motions will probably deform accordingly, and not change topologically. Some methods exhibit these properties (for instance, DMPs), but not necessarily when learning manifolds.

These observations are not in and of themselves novel - lots of work has noticed these things. But very little work has tried to tackle them explicitly and from first principles, and design solutions which exploit the mathematical structure of these properties. And I haven’t really seen much / any work which tries to rope in group theory / formal symmetry properties into trajectory learning. So to me, this is quite original for robot learning - I learned something by reading the paper, and I think the community would benefit from thinking about these properties with a more rigorous lens.

# Quality

## Theoretical

As far as I can tell, the theoretical contributions of this work are sound. 

One question: is the independence regularization term from prior work? Or designed for this work?

## Empirical

The two examples are reasonably well-executed. Both the toy task and the real-world robot task have rigorously-defined symmetries, and

However, I guess I have the following issue with the baselines: if a task is simple enough to write down all the precise symmetries for, then sufficiently rich neural network models and a sufficiently large set of demonstrations should be able to represent these symmetries as well. More recent work than TC-VAE has shown impressive ability to represent multimodal manifolds (i.e. [1]), so I’d be curious how much of a benefit leveraging these simple explicit symmetries actually has over more recent techniques.

[1] Diffusion Policy: Visuomotor Policy Learning via Action Diffusion

# Clarity

My biggest issue with the paper is that the mathematical treatment of symmetry was a bit rushed + inaccessible. For this community, it’s unlikely that people will be familiar with the differences between group operations and group actions (and may conflate the two), when in fact that distinction is actually quite important to understanding what the symmetry operators are doing. The symmetry operators should at the very least have a larger diagram which relates the notation to an actual example. Figure 2 should be improved, made larger, and related to the operators better. Something with transition arrows like found on [1]. Also, section 3.2 doesn’t have much intuition, and it’s really not clear intuitively how h^bar can be constructed or is invertible. It would be really, really useful to incorporate the toy example here while explaining it.

There isn’t really enough of a related work section. If it doesn’t fit in the main paper, it should be put in the appendix and referenced in the main paper.

[1] https://en.wikipedia.org/wiki/Symmetry_group

# Significance

Another issue I have is in the significance of the work given the assumptions. I’m not as concerned with the homeomorphism assumption as I am with the assumption that mathematically enumerating the symmetries in arbitrary robot learning problems is simple or even possible. In most real-world problems, there are only rough/fuzzy/approximate symmetries that can be exploited, and it would very difficult to write down these symmetries, let alone define an h^bar which can be used to construct an equivariant function under those symmetries. Furthermore, if you’re trying to solve a CLASS of tasks with different symmetries, it’s unclear how you’de represent that in this framework without explicitly enumerating each element in that class. For instance, consider hanging a mug on a rack - there are discrete and rotational symmetries which may be different for different mugs and racks, particularly if the geometries are rather different.

# Relevance

The work is undeniably relevant to robot learning literature - we deal with symmetries very frequently, and many works fail to address them at all, let alone with mathematical rigor. I think many people will find this treatment very interesting + inspiring in this setting.

# Limitations

The authors do an adequate job addressing the two assumptions (homeomorphism, ease of defining h^bar). However, they don’t really mention how the method might be used in a situation where you have a set of symmetry groups you might want one method to solve. This puts a pretty hard cap on the direct utility of this method in real-world learning scenarios.

Not sure if this is a limitation per se, but the homeomorphic assumption is not particularly elegant when it comes to predicting trajectories which are time-parametereized. For instance, stretching a short trajectory to a long trajectory with the (presumably) same set of time-indexed waypoints doesn’t really make so much sense for control. Instead, would be interesting to see if the method could be adapted to some kind of time-invariant trajectory shape through configuration space.

Finally, the method doesn’t really make any discussion of manifold boundaries, which are quite relevant for many real robot tasks.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This paper offers a compelling formal perspective on handling symmetry in robot learning tasks. While I'm not entirely sure about the applicability of this method to a broader range of more-complicated robot problems because it is difficult to exactly satisfy the assumptions, I certainly learned something by reading this paper and believe that the CoRL community would benefit from considering the implications of the theoretical treatments of symmetry/equivariance.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary_of_paper: This paper proposes learning a continuous manifold of trajectories, referred to as Motion Manifold Primitives (MMP), to achieve a specified task. Compared to single trajectories, MMP is able to address unforeseen constraints and obstacles. The authors further propose EMMP, which additionally accounts for equivariant transformations through using an invariant encoder and an equivariant decoder. The proposed methods were applied to synthetic and real-world robot experiments and compared against exiting manifold primitive methods, with EMMP showing superior results. The authors demonstrated that the invariant encoder and equivariant decoder are key factors for performance improvement through ablations and swapping different network architectures.
review: The paper is well written, the derivations are thorough, and the proposed method is well supported with simulation and real-world results. Ample comparison and ablation of the methods are provided.

The appendix and supplementary video provide detailed information about the submission and are both helpful for understanding the paper.

Please refer to the “questions for rebuttal” section for more details.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This paper presents a method that is rigorously justified, the results are favorable, and the comparisons are comprehensive. I recommend accepting this paper to CoRL 2023.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary_of_paper: This paper focues on modeling multiple solution trajectories for one task, using movement pritimives and latent manifold.


$\textbf{Updated review after rebuttal}$:

The paper proposed a novel method to learn the trajectory manifold by using the equivariant nature of movements. By constructing the invariant encoder and equivariant decoder strucures, this proposed method is able to capture the equivariant motion manifold primitives. During the rebuttal, the authors have significantly improved the readbility of the paper, makeing it smooth to read and technically sound.
review: Unfortunately, I found it challenging to comprehend the technical aspects of the paper, even though I believe I possess adequate knowledge in robot learning and movement primitives. My reading stopped at Line 93 on page 3 because I became completely lost. The primary reasons for my inability to continue are as follows:

The paper heavily relies on numerous assumptions without providing compelling justifications for their reasonability. Throughout the paper, the phrases "We assume" and "XXX is assumed to be" appear approximately 20 times, particularly on page 2. However, it remains unclear to me why these assumptions are necessary, making it difficult to assess the correctness of the proposed techniques. For instance, I came across an example from Line 53 to Line 56 that discusses the symmetry transformation, where the reasoning behind such an assumption was unclear.

The mathematical descriptions appear disconnected and disjointed, resembling a collection of random sentences. In particular, the lack of coherence between sentences from Line 88 to Line 93 makes it challenging to establish the causality between assumptions and the corresponding mathematical inferences.

To enhance the paper's clarity and credibility, I suggest addressing these issues and providing clearer justifications for the underlying assumptions, as well as improving the flow and coherence of the mathematical descriptions.


$\textbf{Updated review after rebuttal}$:

I appreciate the effort of the authors to improve the readability of the paper. After reading the revised version, I see that my main concern has been addressed. I'm in favor of this paper now.

In the first version, I struggled to understand the theory because of missing explanations and assumed prior knowledge. But in this revised paper:

- The authors added an introduction to the Geometric Preliminaries, which helped me grasp the main theory.
- They've rewritten the methods section and updated the figures, offering a clearer explanation of the method.
- They removed some unnecessary assumptions, making the theory much sounder.

With these changes, I can now see the paper's novelty and contribution. The theory is well-presented now, even for readers without much knowledge in the trajectory manifold. I think the method can be applied to many robot learning tasks. Despite my strong rejection earlier, I'm now leaning towards a strong accept.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I regret to express my reservation about recommending this paper as I encountered considerable difficulty comprehending the technical details. The abundance of assumptions presented throughout the paper lacks adequate and reasonable explanations, contributing to the overall confusion in the writing.


$\textbf{Updated review after rebuttal}$:

I recommend this paper now, as the revised version has improved the readability a lot. After breaking through the bottleneck of the technical part, I found the idea of the paper is very interesting and should be capable to generalize to a lot of similar tasks.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary_of_paper: This paper proposes to use a manifold learning technique to learn manifolds representing the set of all feasible trajectories that successfully execute a task (given a parameter task $\tau$).

Learning the task for all parameters $\tau$ can be very data-intensive. Still, the authors propose some techniques to increase data efficiency and show that their technique is feasible whenever the manifolds (parametrized by $\tau$) are homeomorphic to each other. 

To learn such manifolds, the paper proposes an auto-encoder architecture that possesses an invariant encoder and an equivariant decoder. Those take into account the symmetries of a task, and this leads to improved generalization and efficiency in learning tasks.

review: This paper makes an interesting contribution to the field of Learning-by-Demonstration. It introduces a manifold learning technique that enables the learning of the entire space of trajectories required to successfully execute a task.

The paper is commendable for its clear illustrations and explanations. It also includes comprehensive experiments that effectively showcase the technique and demonstrate how to leverage the task symmetries using the proposed framework.

However, to ensure acceptance, a few minor issues need to be addressed.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The reviewer finds the paper interesting, with clear presentation and high impact in robotics and machine learning.
review_title: null

Meta-review: The paper presents an imitation learning framework to model a continuous manifold of demonstrated trajectories. To address the issue of the limited data, the proposed method leverages symmetries in robot tasks.
While issues of the presentation in the paper were raised in the initial review, the presentation was significantly improved after the rebuttal. 
As the reviewers agreed to accept the paper, AE recommends the acceptance of the paper.
Decision: Accept (Poster)