Paper: Gaitor: Learning a Unified Representation for Continuous Gait Transition and Terrain Traversal for Quadruped Robots (ICLR-2024)

Reviewer comment (Experiments):
In this paper, there are few works to compare. Although it is firstly to do the task, there should be some way to compare the proposed methods with others, or it is hard to evaluate the performance.

Author response:
Dear Reviewer keBn,

Thanks for taking the time to look over the revised manuscript.

In the Results section, we draw comparisons between other methods in Table 1. We see that there are no other methods which are able to learn a compact representation for locomotion with which continuous gait transitions are automatically generated. Therefore, we are unable to directly compare our methods as they cannot produce this kind of behaviour. 

Our method differs considerably from prior work since our focus is to discover new intermediate gaits from distinct gait types such as trot, crawl and pace. Therefore, we present a representation learning methodology to infer gait transitions. We learn this representation using distinct gaits and then discover the gait transitions by exploring the learnt representation. This is vastly different from other works which learn policies for multiple gaits using reinforcement learning (RL) and in general leverage prior knowledge to learn a policy. For example, Yang et al., 2021 uses a predetermined contact schedule and cannot be used to discover new intermediate skills. Another method Locomotion-Transformer uses a large model with sufficient capacity to a multi-modal policy for locomotion. This method is an RL method and must explore the locomotion workspace to train the policy. Once trained, no new skills can be discovered since this method does not learn an explicit representation for locomotion.

As such, we show the effectiveness of Gaitor by exploring the learnt latent-representation to explain the gait switching mechanism in latent space; we evaluate the performance of the perceptive system whilst climbing steps in the real-world; we show that the planner exploits the structure of the latent space to adjust the swing characteristics to help the robot climb steps; and finally we show that all the trajectories even the discovered gait transitions are dynamically feasible. We would be happy to consider augmenting this evaluation if there are specific aspects you could point us to (together with suitable baselines) which you feel might add value.

Kind regards,

Anon.

Reviewer comment (Writing):
I don't understand the approach section. It would be great if the authors could make the description clearer. See questions below.

Author response:
We have focused our attention on the presentation of the revised paper. In particular, we have restructured the methods section to improve the clarity of the approach and significantly revised Fig 1 and 2 so that the former acts as a deployment diagram, while the latter explains the training regime much more clearly. The revisions that we have made to our paper are in blue in the updated paper.

We have significantly improved the clarity of the paper and ask you to revisit our contributions. Our work, Gaitor, is a representation learning method and does not use RL or exploration to solve locomotion problems. Instead, we learn a compact 2D representation for locomotion, which is trained using distinct gaits. However, we discover that the approach exploits the phase correlations between the robot’s legs to create the representation meaning that, through traversal through the space, intermediate gaits are automatically discovered. To the best of our knowledge, this is the first method capable of generating novel intermediate gaits and does this without the use of RL or exploration of the state space. Furthermore, we show that these continuous transitions between gaits are dynamically feasible by deploying on the real Anymal C platform. We ask for guidance as to how we can strengthen the revised manuscript so that it can showcase the contributions of our work.

Reviewer comment (Writing):
"All pose orientations are represented in tangent space." Can you specify what you mean here?

Author response:
We explain in the revised manuscript that the tangent space is a representation for the Lie-algebra of 3D rotations and is commonly used for robot control and state estimation. We include two citations as examples of how it is used in the wider literature also. Expressing coordinate transforms in the tangent-space has the benefit that non-linear composition of coordinate frames becomes linear vector algebra in the tangent space.

Reviewer comment (Theory):
I don't quite understand why it is called a VAE instead of just a neural network that maps input to output. My understanding is that a VAE will try to reconstruct the input at its output, but that does not seem to be the case here. Perhaps this is why I am very confused when reading the paper. My understanding is the proposed "VAE" learns to project the input to a latent space, which is then used to reconstruct the output (which is different from the input). VAE is also used when given the same input to generate multiple plausible outputs, in the context of robots, say given the same user command velocity, generates different gaits that can follow this velocity. I am not sure where is the variational part in the proposed VAE.

Author response:
There are multiple reasons why we use a Variational Auto-Encoder (VAE) instead of an Auto-Encoder (AE). The first is that during training, the generative nature of the training process compresses our latent space meaning that we find a compact 2D representation for locomotion. This is crucial as properties of locomotion such as swing height and length are disentangled within the 2D space - while these are not explicitly varied parameters in the training set. This means we can solve for any locomotion problem using a trajectory in the 2D manifold. Complete details of this discussion can be found in prior work VAE-Loco. Thus, we refer the reader to this paper in Section 3.2, which discusses these findings in our paper.
We use our learnt 2D latent-space to plan locomotion trajectories within. As we mention in the revised methods section 3, the latent space forms a solution space in which we select a single solution by traversing the space. Indeed, we are using the VAE to learn this manifold and as mentioned the generative training paradigm ensures that this space contains only the principal components necessary to reconstruct the locomotion trajectories.

Reviewer comment (Writing):
I am not sure how the different components are used after training. My naive guess is the past trajectory is fed into the VAE, and then outputs the future trajectory for robot control. The encoding of the inputs can be modified by the planner to adjust gaits. It would be nice to make it clearer how the system works.

Author response:
We have clarified the deployment of Gaitor in two ways. Firstly, we have revised section 3.4 to explain this much more clearly. Secondly, we have revised Fig. 1 so that it explains the deployment process more clearly. Fig 1 is split into three panels which explain inference, planning and construction of the trajectories.

Reviewer comment (Experiments):
Why is an additional planner needed to deal with terrains since terrain is already an input to the decoder and the training data for the decoder and planner are the same (or could be the same)? Maybe an ablation study is needed?

Author response:
We clarify the contribution of the learnt planner in the revised Results Section 4. The planner exploits the latent-space structure  and is used to adjust the swing characteristics in response to terrain. Additionally, the modulation of the trajectory in latent-space is evident when we encode the expert trajectories.

Reviewer comment (Experiments):
It is claimed that prior methods do not learn a shared representation between skills. However, Caluwaerths'23 proposes Locomotion-Transformer, which in fact does this, and reports similar results to this paper. Comparing to Locomotion-Transformer would help put the results in context.

Author response:
We also contextualise the contributions of this work in relation to current art in particular Locomotion-Transformer. Gaitor is a representation learning method and does not use RL or exploration to solve locomotion problems. Instead, we learn a compact 2D representation for locomotion, which is trained using distinct gaits. However, we discover that the approach exploits the phase correlations between the robot’s legs to create the representation meaning that, through traversal through the space, intermediate gaits are automatically discovered. To the best of our knowledge, this is the first method capable of generating novel intermediate gaits and does this without the use of RL or exploration of the state space. Furthermore, we show that these continuous transitions between gaits are dynamically feasible by deploying on the real Anymal C platform.

Reviewer comment (Presentation):
The presentation of the method is poor.

Author response:
We have focused our attention on the presentation of the revised paper. In particular, we have restructured the methods section to improve the clarity of the approach and significantly revised Fig 1 and 2 so that the former acts as a deployment diagram, while the latter explains the training regime much more clearly. The revisions that we have made to our paper are in blue in the updated paper.

Reviewer comment (Presentation):
On Page 4, the method diagram is described in text, which would be more easily explained in a figure. Figure 1 presumably depicts the same information but is not helpful for understanding the method since Sec 3.1 does not reference the figure. The reader needs to guess the correspondence between the text and the figure. Hafner'20 is cited as an example of good presentation in this context.

Author response:
We have focused our attention on the presentation of the revised paper. In particular, we have restructured the methods section to improve the clarity of the approach and significantly revised Fig 1 and 2 so that the former acts as a deployment diagram, while the latter explains the training regime much more clearly. The revisions that we have made to our paper are in blue in the updated paper.

Reviewer comment (Experiments):
Why is it important that the method doesn't use gait phases as an inductive bias? A comparison to Yang'21 that showcases this difference between methods would strengthen the paper.

Author response:
As mentioned in the paper, few works are able to generalise over multiple gait types. The methods which do tend to use reinforcement learning to learn a policy using explicit inductive biases. For example, Yang et al., 2021 uses a predetermined contact schedule to train a gait switching policy and operates in task-space. Our work is able to discover a joint-space trajectory and contact schedule for gait transition purely from the representation learnt using examples of the distinct gaits.

Reviewer comment (Experiments):
A discussion of a comparison to Locomotion-Transformer is crucial.

Author response:
Locomotion-transformer uses a model of suitable capacity to learn a multi-modal distribution conditioned on a terrain token to produce a policy suitable for traversing different types of terrain. This is not a representation learning model and uses RL during training to learn policies. In contrast, once our locomotion representation is learnt, we can generate new intermediate gaits just by traversing the structure of this space.

Reviewer comment (Presentation):
Overall, the paper is promising but I am unable to recommend accept due to poor presentation.

Author response:
As suggested by a number of the original reviews, we have endeavoured to improve the presentation by restructuring the methods section. The contributions, scientific content, overall structure, experiments in both simulation and hardware, introduction and abstract are unchanged. We therefore argue that the nature of the submission has not changed and the reviewer’s new stance, that our submission should no longer be considered for publication is not, in fact, justified. We note that none of the other reviewers have raised this as a concern and propose that this sort of revision is exactly the point of this phase of the review process. 

We request that the Area Chair mediate this concern.