Venue: NeurIPS 2021 Track Datasets_and_Benchmarks Round2

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: The paper introduces a high-performance training platform for a wide variety of reinforcement learning tasks. The key innovation is to run training end-to-end on GPU, reducing CPU-bound bottlenecks (for instance, when a simulation was on GPU but physics state is on CPU), achieving 2-3 orders of magnitude speedup in training times. Isaac Gym provides an optimized implementation which vectorizes observations and actions on GPU allowing to take advantage of the parallelization provided by the simulator.

The speedups are measured and reported on a wide variety of simulation RL tasks, including Ant, Humanoid, and Shadow Hand. The experiments are run on a single A100 GPU.
strengths: Providing faster, more efficient implementation would lower barrier for RL simulation tasks, and achieve new results.

Thorough evaluation on various RL tasks and simulators.

An API to interface with existing work in this domain.
weaknesses: Comparison to CPU clusters is not quantified: can authors compare costs of A100 GPU vs a CPU cluster, or perhaps compare FLOPs?

GPU accelerations introduced in Isaac Gym might not provide perf improvements for simpler tasks and fewer environments.
correctness: Yes
clarity: The paper is well written and easy to follow.
relation to prior work: The comparison to prior work is clear.
additional feedback: L120: “changing the number of environments from 256 to 8192 -- an increase by 5 orders of magnitude --” must by a typo here?
review_title: High-performance training platform for reinforcement learning tasks

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: summary and contributions: The paper presents Isaac Gym, a gpu accelerated physics simulation that is used for training policies for robotics via reinforcement learning. Isaac gym substantially decreasing the time to solution for RL tasks by running and optimizing the physics simulation for the GPU and removing any CPU bottlenecks caused by the communication between simulation and training. The paper also demonstrates that their simulator is capable of training policies that can transfer to real robots in real environments.

strengths: 1. Characterization of simulation performance in isolation (# of environments) on 3 tasks
2. Numerous benchmarks of RL performance on a variety of standard tasks.
3. Substantial speedups over the existing state-of-the-art on all benchmarks.

weaknesses: There is insufficient explanation of the cause of the non-deterministic behavior observed on the shadow hand experiment. Is it due to the domain randomization is some way?
correctness: Yes the submission is correct and the experimental design is appropriate and properly performed.
clarity: The paper is well written.

relation to prior work: The paper provides a satisfactory discussion of the related work.

additional feedback: The figures in section 5 are difficult to read.
review_title: Great contribution to the robotics community

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: In this paper, the authors designed and implemented Isaac Gym, which is a powerful system that enables the users to train policies for a wide variety of robotics tasks entirely on GPU.

This paper is well written. The authors studied a very important issue. Isaac Gym is a very impressive system for Robot learning. The design I like most is the Tensor API in Python providing direct access to physics buffers by wrapping them into PyTorch tensors without going through any CPU bottlenecks. This can significantly improve the performance and also be very easy to use. It will also support a variety of environment sensors - position, velocity, force, torque, etc.

However, there are also some concerns in this paper. As mentioned by the authors, maximum acceleration of the training process can be achieved only when simulating thousands of environments in parallel for challenging tasks. For simpler tasks or fewer environments, GPU accelerated end-to-end simulation may provide only a minor performance improvement, or none at all. In some cases, fully deterministic simulation of all environments may not be possible.

strengths: This paper is well written. The authors studied a very important issue. Isaac Gym is a very impressive system for Robot learning. The design I like most is the Tensor API in Python providing direct access to physics buffers by wrapping them into PyTorch tensors without going through any CPU bottlenecks. This can significantly improve the performance and also be very easy to use. It will also support a variety of environment sensors - position, velocity, force, torque, etc.
weaknesses: However, there are also some concerns in this paper. As mentioned by the authors, maximum acceleration of the training process can be achieved only when simulating thousands of environments in parallel for challenging tasks. For simpler tasks or fewer environments, GPU accelerated end-to-end simulation may provide only a minor performance improvement, or none at all. In some cases, fully deterministic simulation of all environments may not be possible.
correctness: Good
clarity: Good
relation to prior work: Good
additional feedback: N/A
review_title: A powerful GPU-based system for Robot Learning

Meta-review: This paper proposes a very performant physics engine for robotics implemented on the GPU. The main benefit of this approach is that it enables end-to-end GPU training, which reduces the CPU to GPU communication and transfers. The speedups are measured on a wide range of RL tasks.

The paper is well-written and the overall feedback from the reviewers were positive. However, reviewers raised some concerns in their reviews. I recommend the authors to take those comments into account and try to address them in the camera-ready version of this paper.
Decision: Accept