Venue: NeurIPS 2022 Track Datasets and Benchmarks

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: summary and contributions: - Trained and release high-quality agents that can track over three hours of MoCap data for a simulated humanoid in the dm_control physics-based environment
- Release MoCapAct, a dataset of these expert agents and their rollouts of MoCap-tracking policies in the dm_control humanoid environment
- Demonstrate applications of training with MoCapAct i.e. re-use with reinforcement learning, motion completion with GPT

strengths: - There are previously no agents publicly available that can track all the MoCap data within dm_control physics-based environment. Releasing high-quality agents could lower the barrier of research.
- Useful and well-documented tools and several applications provided.

weaknesses: - I’m concerned about the scale of the dataset. The MoCap dataset integrates motion sequences from only the CMU Mocap dataset. I think there are a number of other MoCap datasets i.e. Human3.6, ZJU-MoCap, SFU, KIT, AMASS. Why was CMU included? 
- It was repeatedly mentioned that training sequential decision-making agents on hours of MoCap data requires significant compute. Therefore, this work released high-quality agents could lower the barrier of research. Perhaps some information about the computational resources or training time could be given to better highlight that importance of MocapAct.
- One of the main focus was on the applications, I was wondering if you can provide a baseline for training with your dataset on these tasks so that future work can have a comparison to?

correctness: NA
clarity: - Well written and concise
relation to prior work: - L79-80: The comparison to prior works are rather scarce. Instead of saying “We have found that while some prior work has released source code and policies”, perhaps more description could be given to elaborate on  what are the prior works and the existing tools, and why “the provided tools scale poorly on large MoCap datasets”.
- There are several large-scale MoCap datasets, only H36M and CMU were mentioned. Why was CMU chosen over the other MoCap datasets? And will there be future plans to release expert rollouts for other datasets? 

additional feedback: - Issue on maintainability and whether more datasets will be added in the future?
- Would it be possible to provide baselines for training with your dataset on the various tasks?
review_title: Paper review for "MoCapAct: A Multi-Task Dataset for Simulated Humanoid Control"

final_score: rating: 6: Marginally above acceptance threshold
review_content: summary and contributions: Though MoCap data could be used to for humanoid control learning, intensive computation is required since MoCap contains only kinematic information with no physical control inputs. To lower the computation barrier, the authors trained and released a high-quality agent to track the MoCap data in a physics-based environment. Furthermore, a dataset named MoCapAct is proposed using this expert agent. The dataset empowers a hierarchical policy for low-level skill learning and high-level task learning. 
strengths: The hierarchical policy for multi-level task learning trained on the newly proposed dataset is inspiring and promising. The idea of reusable low-level policy is helpful in reducing the computation burden.
weaknesses: 1. Some references are missing. For the field of motion imitation learning, [this NeurIPS 2020 paper](https://arxiv.org/pdf/2006.07364.pdf) should be taken into discussion. Also, the methodology of MoCapAct seems to be augmenting the original motion sequences in dm_conrol environment, which is quite similar to [this paper](https://arxiv.org/abs/2203.09116). Adding proper discussion would be helpful.
2. I have some concerns on the significance of MoCapAct. The proposed dataset seems like an augmentation upon CMU MoCap, while the applications are also not unique. 
3. As I understand, MoCapAct is composed by clip experts and augmented motion sequences from the expert rollouts. While in the experiment section, the latter could be replaced as raw motion sequences. I would like to know how much improvement the expert rollouts bring. 
4. The diversity of expert rollouts is another concern that bothers me. Wouldn't there be too many similar sequences among multiple rollouts of the same expert?
4. In the experiment section, most comparison are conducted between self-defined baselines, and using metrics which are not universal. Using some universal metrics like mean angle error, final angle error, MPJPE might be helpful both for understanding and for better comparison with previous methods.

----------------- After rebuttal --------------------

The authors' response basically addressed my concerns. Therefore, I'm leaning to accept it.
correctness: I believe the claims are correct.
clarity: The paper is well-written.
relation to prior work: Please refer to weakness.
additional feedback: -
review_title: Review on paper 55

final_score: rating: 6: Marginally above acceptance threshold
review_content: summary and contributions: 
This paper proposes a humanoid dataset for the RL expert agents and their rollouts containing proprioceptive observations and actions based on the motion capture data of humans. Compared with existing datasets, the paper can learn low-level motor skills. The dataset is demonstrated by training a multi-clip tracking policy to learn new high-level tasks and training a GPT model to generate motion. 

strengths: This paper has good coverage of datasets from the motion capture and reinforcement learning literature.
The experimental details seem clearly written and well-explained.
The multi-clip policy shows a good result of what we should expect from motion capture data.

weaknesses: The code presented with the paper seems to be lacking proper documentation on the environment. More specifically, it would be great if authors include a few lines in documentation explaining how someone completely unfamiliar with the code can start using their environment to train high-level tasks. And how can someone reproduce results in the paper with the scripts provided?
correctness: 
Seems correct.

clarity: Fairly clear.
relation to prior work: The paper discussed and compared with the prior work sufficiently.

additional feedback: Nothing particular.
review_title: MoCapAct review

final_score: rating: 5: Marginally below acceptance threshold
review_content: summary and contributions: The paper mainly aims to lower the barrier for research on humanoid control simulation. With a physics-based environment, they train agents that can track large-scale MoCap data for a simulated humanoid. The proposed MoCapAct dataset consists of the trained agents and their rollouts. Two applications are demonstrated with the build MoCapAct dataset, namely Multi-clip tracking policy and motion completion with GPT.
strengths: [S1] The idea to keep the trained agents and use their rollouts for downstream tasks is interesting.

[S2] Two applications are demonstrated in detail to show the effectiveness of the produced dataset.

[S3] The authors provide dataset and code to facilitate research on humanoid control research
weaknesses: [W1] Previous attempts also tried to resolve training control policies from scratch, for example, ASE [Peng et al 2022] has shown that a single pre-trained model can be effectively applied to perform a diverse set of new tasks.  Any quantatitve comparison with these prior works?

[W2] It seems that most shown tasks are walking or running, but DeepMimic [Peng et al 2018] is able to perform complex motion like back flip. Any complicated actions exist in MoCapAct?

[W3] For motion completion task in sec. 5.2, it is not very clear the benefits of using MoCapAct compared with other motion completion approaches. Besides, as not all motion completion methods are based on RL, will other methods like [3, 4] benefit from MoCapAct?

---
Reference

[1] Peng, Xue Bin, et al. "ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters." arXiv preprint arXiv:2205.01906 (2022).

[2] Peng, Xue Bin, et al. "Deepmimic: Example-guided deep reinforcement learning of physics-based character skills." ACM Transactions On Graphics (TOG) 37.4 (2018): 1-14.

[3] Tevet, Guy, et al. "MotionCLIP: Exposing Human Motion Generation to CLIP Space." arXiv preprint arXiv:2203.08063 (2022).

[4] Harvey, Félix G., et al. "Robust motion in-betweening." ACM Transactions on Graphics (TOG) 39.4 (2020): 60-1.
correctness: The produced dataset and the exprimental setups are sound.
clarity: The paper is easy to follow
relation to prior work: Important prior works have been discussed in related works
additional feedback: NA
review_title: A useful pretrained mocap dataset but some concerns are left for experiments

final_score: rating: 6: Marginally above acceptance threshold
review_content: summary and contributions: This paper introduces a dataset containing policies and their rollouts for MoCap dataset. The trained policies are able to track the majority of the MoCap snippets. In principle, this dataset could be used to learn high-level other tasks. In this paper, the authors demonstrate several applications from this dataset, including using the rollouts to train a multi-clip policy and GPT policy, although I suspect it's flexible enough to evaluate various other things as well.
strengths: [S1] The dataset could be flexibly used to investigate policy learning for a simulated humanoid in the dm_control environment. The provided expert and its rollouts eliminate the need for extensive computation to train from scratch.

[S2] The discussion of related methods is thorough, although I think there is still a need for a clear comparison of the method in the paper with the previous work (see below).

[S3] The dataset is open source, which provides detailed instructions and demos


weaknesses: [W1] What's the essential difference between the clip expert in this paper and the previous solutions? It seems that the policy learning and the stochastic rollouts used in this paper are basically an integrated use of several previous works. Is the main difference of this paper to reproduce and open source the previous method in this specific dm_control environment?

[W2] If not, I think the authors should show some comparisons with baselines, e.g. previous methods do not track all MoCap data well.

[W3] The analysis and evaluation for expert rollouts need to be more detailed, such as an analysis of the diversity of these rollouts, and their quality under the diversity. I don't see such quantitative analysis of this in the paper. 

[W4] A related weakness is that the videos of noisy rollouts on the webpage do show a limited variety of rollouts, but some noticeable jitter in motion can be seen. 
correctness: Yes, the dataset is constructed in a sound way and the evaluation methods and experiment design are appropriate and performed correctly.
clarity: Yes, the paper is overall well written and easy to understand. Some recommendations: 

[CL1] As mentioned in W3, I would like to see some quantitative analysis of noisy rollouts.

[CL2] Fig.8 is a bit confusing to me, although I can understand it from the text that explains it. I think the examples on the webpage might be clearer
relation to prior work: The authors clearly discussed how the proposed dataset differs from previous contributions.

[R1] As mentioned in [W1], however, I'm particularly interested to know the superiority of the policies provided in this paper
additional feedback: N/A
review_title: A dataset of MoCap policies and rollouts

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: The authors propose MoCapAct, a dataset of these expert agents and their rollouts containing proprioceptive observations and actions. Considering the large compute budgets, their study facilitates  the use of MoCap data in humanoid control research. Furthermore, they use MoCapAct to train an autoregressive GPT model and show that it can perform natural motion completion given a motion prompt.
strengths: The dataset contains MoCap-tracking policies in high quality, which can track 3.5 hours of recorded motion from CMU MoCap. Additionally, the experimental results illustrate MoCapAct’s usefulness for learning diverse motions. More interestingly, the dataset are utilized to train a GPT network, which demonstrates the effectiveness of big model. 
weaknesses: The experiments part said you use PPO algorithm to train the experts. What about the off-policy algorithm? I hope you can show more comparisons with different algorithms. Besides, I think your work can be extended into other environments with extra objects. The invention of other objects makes the learning more challenging in the future. To sum up, I expect to see a dataset with more sufficient scenarios in the future.
correctness: The construction of the dataset and evaluating methods are technically sound.
clarity: The paper is well written and easy to follow.
relation to prior work: The discussion about prior works are clear.
additional feedback: None.
review_title: A good benchmark for humanoid control research 

Meta-review: metareview: In this paper, the authors propose a multi-task dataset for simulated humanoid control. Reviewers had concerns about the scale of this benchmark and its significance, many of which are addressed during the author response periods. We encourage the authors to further address the remaining feedback, especially those from EaWT.

Overall, I would recommend Accept.
confidence: 3: The area chair is somewhat confident
Decision: Accept