Original: D A XB ENCH : B ENCHMARKING D EFORMABLE O BJECT M ANIPULATION WITH D IFFERENTIABLE P HYSICS
Revised: D A XB ENCH : B ENCHMARKING D EFORMABLE O BJECT M ANIPULATION WITH D IFFERENTIABLE P HYSICS

Original: A BSTRACT
Revised: A BSTRACT

Original: Deformable object manipulation (DOM) is a long-standing challenge in robotics and has attracted significant interest recently.
Revised: Deformable object manipulation (DOM) is a long standing challenge in robotics and has attracted significant interest recently

Original: This paper presents DaXBench, a differentiable simulation framework for DOM.
Revised: This paper presents DaXBench, a differentiable simulation framework for DOM.

Original: In contrast with existing workthat often focuses on a specific type of deformable objects, DaXBench supports fluid, rope, cloth ...; it provides a general-purpose benchmark to evaluate widely different DOM methods, including planning, imitation learning and reinforcement learning.
Revised: While existing work often focuses on a specific type of deformable objects, DaXBench supports fluid, rope, cloth ...; it provides a general-purpose benchmark to evaluate widely different DOM methods, including planning, imitation learning, and reinforcement learning.

Original: DaXBench combines recent advances in deformable object simulation with JAX, a high-performance computational framework.
Revised: DaXBench combines recent advances in deformable object simulation with JAX, a high-performance computational framework.

Original: All DOM tasks in DaXBench are wrapped with  OpenAI Gym APIs, forfast and easy integration with DOM algorithms.
Revised: All DOM tasks in DaXBench are wrapped with the OpenAI Gym API for easy integration with DOM algorithms.

Original: We hope that DaXBench provides to the research community a comprehensive, standardized benchmark and a valuable tool that supports the development and evaluation of new DOM methods.
Revised: We hope that DaXBench provides to the research community a comprehensive, standardized benchmark and a valuable tool to support the development and evaluation of new DOM methods.

Original: The code and video are available online ∗ .
Revised: The code and video are available online ∗ .

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deformable object manipulation (DOM) is a crucial area of research with a wide range of applications, from householdscenarios (Maitin-Shepard et al., 2010; Miller et al., 2011;
Revised: Deformable object manipulation (DOM) is a crucial area of research with broad applications, from household (Maitin-Shepard et al., 2010; Miller et al., 2011;

Original: Ma et al., 2022) to industrial settings (Miller et al., 2012; Zhu et al., 2022).
Revised: Ma et al., 2022) to industrial settings (Miller et al., 2012; Zhu et al., 2022).

Original: To aid in algorithm development and prototyping, several DOM benchmarks (Lin et al., 2021; Huang et al., 2021) have been developed using deformable object simulators.
Revised: To aid in algorithm development and prototyping, several DOM benchmarks (Lin et al., 2021; Huang et al., 2021) have been developed using deformable object simulators.

Original: However,due to the high dimensionality of state and action spaces in DOM tasks, creating a generalizable and effective policy is still a significant challenge.
Revised: However, the high dimensional state and action spaces remain a significant challenge to DOM.

Original: Differentiable physics is a promising direction for developing control policies for deformable objects.
Revised: Differentiable physics is a promising direction for developing control policies for deformable objects.

Original: It implements physical laws as differentiable computational graphs (Freeman et al., 2021;
Revised: It implements physical laws as differentiable computational graphs (Freeman et al., 2021;

Original: Hu et al., 2020), enabling the optimization of control policies with analytical gradients and therefore improving sample efficiency.
Revised: Hu et al., 2020), enabling the optimization of control policies with analytical gradients and therefore improving sample efficiency.

Original: Recent studies have shown that differentiable physics-based DOM methods can benefit greatly from this approach (Chen et al., 2022; Hu et al., 2020; Huang et al., 2021; Xu et al., 2022; Heiden et al., 2021).
Revised: Recent studies have shown that differentiable physics-based DOM methods can benefit greatly from this approach (Huang et al., 2021; Heiden et al., 2021; Xu et al., 2022; Chen et al., 2022).

Original: Therefore, to facilitate fair comparison and further advancement of DOM techniques, it is important to develop a general-purpose simulation platform that accommodates these methods.
Revised: To facilitate fair comparison and further advancement of DOM techniques, it is important to develop a general-purpose simulation platform that accommodates these methods.

Original: We present DaXBench, a differentiable simulation framework for deformable object manipulation (DOM).
Revised: We present DaXBench, a differentiable simulation framework for deformable object manipulation (DOM).

Original: In contrast to current single-task benchmarks, DaXBench encompasses a diverse range of object types, including rope, cloth, liquid, and elasto-plastic materials.
Revised: In contrast with current single-task benchmarks, DaXBench encompasses a diverse range of object types, including rope, cloth, liquid, and elasto-plastic materials.

Original: The platform includes tasks with varying levels of difficulty and well-defined reward functions, such as liquid pouring, cloth folding, and rope wiping.
Revised: The platform includes tasks with varying levels of difficulty and well-defined reward functions, such as liquid pouring, cloth folding, and rope wiping.

Original: These tasks are designed to support high-level macro actions and lowlevel controls, enabling comprehensive evaluation of DOM algorithms with different action spaces.
Revised: These tasks are designed to support high-level macro actions and low-level controls, enabling comprehensive evaluation of DOM algorithms with different action spaces.

Original: We benchmark eight competitive DOM methods across different algorithmic paradigms, including sampling-based planning, Reinforcement Learning (RL), and Imitation Learning (IL).
Revised: We benchmark eight competitive DOM methods across different algorithmic paradigms, including sampling-based planning, reinforcement learning (RL), and imitation learning (IL).

Original: For planning methods, we consider CrossEntropyMethod-MPC (CEM-MPC)
Revised: For planning methods, we consider model predictive control with the Cross Entropy Method (CEM-MPC)

Original: (Richards, 2005), differentiable MPC (Hu et al., 2020), and a combination of the two.
Revised: (Richards, 2005), differentiable model predictive control (Hu et al., 2020), and a combination of the two.

Original: For RL domains, we consider Proximal Policy Optimization (PPO) (Schulman et al., 2017) with non-differentiable dynamics, and Short-Horizon Actor-Critic (SHAC)
Revised: For RL domains, we consider Proximal Policy Optimization (PPO) (Schulman et al., 2017) with non-differentiable dynamics, and Short-Horizon Actor-Critic (SHAC)

Original: (Xu et al., 2022) and Analytic Policy Gradients (APG) (Freeman et al., 2021) with analytical gradients.
Revised: (Xu et al., 2022) and Analytic Policy Gradients (APG) (Freeman et al., 2021) with analytical gradients.

Original: For IL, Transporter networks (Seita et al., 2021) with non-differentiable dynamics and Imitation Learning via Differentiable physics (ILD) (Chen et al., 2022) are compared.
Revised: For IL, Transporter networks (Seita et al., 2021) with non-differentiable dynamics and Imitation Learning via Differentiable physics (ILD) (Chen et al., 2022) are compared.

Original: Our experiments compare algorithms with and without analytic gradients on each task, providing insights into the benefits and challenges of differentiable-physics-based DOM methods.
Revised: Our experiments compare algorithms with and without analytic gradients on each task, providing insights into the benefits and challenges of differentiable-physics-based DOM methods.

Original: DaXBench implements a deformable object simulator, DaX, which combines recent advancements in deformable object simulation algorithms (Xu et al., 2022; Chen et al., 2022) with the highperformance computational framework JAX (Bradbury et al., 2018).
Revised: DaXBench provides a deformable object simulator, DaX, which combines recent advances in deformable object simulation algorithms (Xu et al., 2022; Chen et al., 2022) with the high-performance computational framework JAX (Bradbury et al., 2018).

Original: This integration allows for efficient auto-differentiation and parallelization across multiple accelerators, such as multiple GPUs.
Revised: This integration allows for efficient autodifferentiation and parallelization across multiple accelerators, such as multiple GPUs.

Original: All task environments are wrapped with  OpenAI Gym APIs (Brockman et al., 2016), enabling seamless integration with DOM algorithms for fast and easy development.
Revised: All task environments are wrapped with the OpenAI Gym API (Brockman et al., 2016), enabling seamless integration with DOM algorithms for fast and easy development.

Original: By providing a comprehensive and standardized simulation framework, we aim to facilitate algorithm development and advance the state of the art in DOM.
Revised: By providing a comprehensive and standardized simulation framework, we aim to facilitate algorithm development and advance the state of the art in DOM.

Original: Moreover, our  results demonstrate that the dynamics modelused in
Revised: Moreover, our experimental results show that the dynamics model in

Original: DaXBench enables direct sim-to-real transfer to a real robot for rope manipulation tasks, demonstrating the potential applicability of our simulation platform to real-world problems.
Revised: DaXBench enables direct sim-to-real transfer to a real robot for rope manipulation, indicating the potential applicability of our simulation platform to real-world problems.

Original:  R ELATED W ORKS 2. D EFORMABLE O BJECT S IMULATORS
Revised:  R ELATED W ORKS 2. D EFORMABLE O BJECT S IMULATORS

Original: Recent advancements in Deformable Object Manipulation (DOM) have been partially attributed to the emergence of DOM simulators in recent years.
Revised: Recent advancements in Deformable Object Manipulation (DOM) have been partially attributed to the emergence of DOM simulators in recent years.

Original: SoftGym (Lin et al., 2021) is the first DOM benchmark that models all liquid, fabric, and elastoplastic objects and introduces a wide range of standardized DOM tasks.
Revised: SoftGym (Lin et al., 2021) is the first DOM benchmark that models all liquid, fabric, and elastoplastic objects and introduces a wide range of standardized DOM tasks.

Original: ThreeDWorld
Revised: ThreeDWorld

Original: (Gan et al., 2020) enables agents to physically interact with various objects in rich 3D environments, but it is non-differentiable and does not support methods based on differentiable physics.
Revised: (Gan et al., 2020) enables agents to physically interact with various objects in rich 3D environments, but it is non-differentiable and does not support methods based on differentiable physics.

Original: On the other hand, several differentiable DOM simulators emerge, such as ChainQueen (Hu et al., 2019), Diff-PDE (Holl et al., 2020), PlasticineLab (Huang et al., 2021), DiSECt (Heiden et al., 2021), and DiffSim (Qiao et al., 2020).
Revised: On the other hand, several differentiable DOM simulators emerge, such as ChainQueen (Hu et al., 2019), Diff-PDE (Holl et al., 2020), PlasticineLab (Huang et al., 2021), DiSECt (Heiden et al., 2021), and DiffSim (Qiao et al., 2020).

Original: However, each simulator specializes in modeling a single type of deformable object and supports only a limited range of object-specific tasks.
Revised: However, each simulator specializes in modeling a single type of deformable object and supports only a limited range of object-specific tasks.

Original: For example,
Revised: For example,

Original: PlasticineLab (Huang et al., 2021) and DiSECt (Heiden et al., 2021) model only the elasto-plastic objects, including tasks such as sculpting, rolling and cutting the deformable objects.
Revised: PlasticineLab (Huang et al., 2021) and DiSECt (Heiden et al., 2021) model only the elasto-plastic objects, including tasks such as sculpting, rolling and cutting the deformable objects.

Original: Since the dynamics of each deformable object are vastly different, the physic engine specialized in modeling one type of deformable object cannot be easily extended to another.
Revised: Since the dynamics of each deformable object are vastly different, the physic engine specialized in modeling one type of deformable object cannot be easily extended to another.

Original: DaXBench bridges this gap by offering a general-purpose simulation framework that covers a wide range of deformable objects and the relevant tasks.
Revised: DaXBench bridges this gap by offering a general-purpose simulation framework that covers a wide range of deformable objects and the relevant tasks.

Original: We provide a fairground for comparing and developing all types of DOM methods, especially the differentiable ones.
Revised: We provide a fairground for comparing and developing all types of DOM methods, especially the differentiable ones.

Original: 2. D EFORMABLE OBJECT MANIPULATION ALGORITHMS
Revised: 2. D EFORMABLE OBJECT MANIPULATION ALGORITHMS

Original: These challenges make it difficult to apply existing methods for rigid object manipulation directly to DOM.
Revised: These challenges make it difficult to apply existing methods for rigid object manipulation directly to DOM.

Original: Methods designed for DOM must be able to handle a large number of degrees of freedom in the state space and the complexity of the dynamics.
Revised: Methods designed for DOM must be able to handle a large number of degrees of freedom in the state space and the complexity of the dynamics.

Original: Despite these challenges, many interesting approaches have been proposed for DOM.
Revised: Despite these challenges, many interesting approaches have been proposed for DOM.

Original: Depending on the algorithmic paradigms, we categorize
Revised: Depending on the algorithmic paradigms, we categorize

Original: DOM methods into three groups: planning, Imitation Learning (IL), and Reinforcement Learning (RL).
Revised: DOM methods into three groups: planning, Imitation Learning (IL), and Reinforcement Learning (RL).

Original: Our discussion below focuses on the general DOM methods for a standardized set of tasks.
Revised: Our discussion below focuses on the general DOM methods for a standardized set of tasks.

Original: We refer to (Sanchez et al., 2018; Khalil & Payeur, 2010) for a more detailed survey on prior methods for robot manipulation of deformable objects.
Revised: We refer to (Sanchez et al., 2018; Khalil & Payeur, 2010) for a more detailed survey on prior methods for robot manipulation of deformable objects.

Original: Reinforcement Learning algorithms using differentiable physics such as SHAC (Xu et al., 2022) and APG (Freeman et al., 2021) are also explored.
Revised: Reinforcement Learning algorithms using differentiable physics such as SHAC (Xu et al., 2022) and APG (Freeman et al., 2021) are also explored.

Original: These methods use the ground-truth gradients on the dynamics to optimize the local actions directly, thereby bypassing the massive sampling for the policy gradient estimation.
Revised: These methods use the ground-truth gradients on the dynamics to optimize the local actions directly, thereby bypassing the massive sampling for the policy gradient estimation.

Original: Imitation Learning is another important learning paradigm for DOM.
Revised: Imitation Learning is another important learning paradigm for DOM.

Original: To overcome the “curse of history”, the existing works such as (Seita et al., 2021; Sundaresan et al., 2020; Ganapathi et al.,2021) overcome the enormous state/action space by learning the low dimensional latent space and
Revised: To overcome the “curse of history”, the existing works such as (Seita et al., 2021; Sundaresan et al., 2020; Ganapathi et al.,

Original: Engine
Revised: Engine

Original: Application
Revised: Application

Original: Differentiability
Revised: Differentiability

Original: Parallelization
Revised: Parallelization

Original: Liquid
Revised: Liquid

Original: Fabric
Revised: Fabric

Original: Elastoplastic
Revised: Elastoplastic

Original: Mixture # of simulations per process
Revised: Mixture # of simulations per process

Original: Multi-Device
Revised: Multi-Device

Original: Training
Revised: Training

Original: PlasticineLab ✗ ✗ ✓ ✗ ✓
Revised: PlasticineLab ✗ ✗ ✓ ✗ ✓

Original: Single ✗
Revised: Single ✗

Original: DiSECt ✗ ✗ ✓ ✗ ✓
Revised: DiSECt ✗ ✗ ✓ ✗ ✓

Original: Single ✗
Revised: Single ✗

Original: DiffSim ✗ ✓ ✗ ✗ ✓
Revised: DiffSim ✗ ✓ ✗ ✗ ✓

Original: Single ✗
Revised: Single ✗

Original: SoftGym ✓ ✓ ✗ ✗ ✗
Revised: SoftGym ✓ ✓ ✗ ✗ ✗

Original: Single ✗
Revised: Single ✗

Original: DaXBench ✓ ✓ ✓ ✓✓
Revised: DaXBench ✓ ✓ ✓ ✓

Revised: [Table]

Original: Multiple
Revised: Multiple

Original: [Table]
Revised: [Table]

Original:  simplifying the primitive actions to only pick-and-place.
Revised: 2021) overcome the enormous state/action space by learning the low dimensional latent space and simplifying the primitive actions to only pick-and-place.

Original: ILD (Chen et al., 2022) is another line of IL method that utilizes the differentiable dynamics to match the entire trajectory all at once; therefore, it reduces the state coverage requirement by the prior works.
Revised: ILD (Chen et al., 2022) is another line of IL method that utilizes the differentiable dynamics to match the entire trajectory all at once; therefore, it reduces the state coverage requirement by the prior works.

Original: Motion planning for DOM has been explored in (Wi et al., 2022;
Revised: Motion planning for DOM has been explored in (Wi et al., 2022;

Original: Shen et al., 2022; Lippi et al., 2020; Nair & Finn, 2019; Yan et al., 2021; Ma et al., 2022).
Revised: Shen et al., 2022; Lippi et al., 2020; Nair & Finn, 2019; Yan et al., 2021; Ma et al., 2022).

Original: These methods overcome the enormous
Revised: These methods overcome the enormous

Original: DoFs of the original state space by learning a low-dimensional latent state representation and a corresponding dynamic model for planning.
Revised: DoFs of the original state space by learning a low-dimensional latent state representation and a corresponding dynamic model for planning.

Original: The success of these planning methods critically depends on the dimensionality and quality of the learned latent space and the dynamic model, which in itself is challenging.
Revised: The success of these planning methods critically depends on the dimensionality and quality of the learned latent space and the dynamic model, which in itself is challenging.

Original: We believe that the development of a general-purpose simulation framework with a differentiable simulator can provide a platform for comparing existing DOM methods.
Revised: We believe that the development of a general-purpose simulation framework with a differentiable simulator can provide a platform for comparing existing DOM methods.

Original: Such a framework enables researchers to gain a better understanding of the progress and limitations of current DOM methods, which in turn can provide valuable insights for advancing DOM methods across all paradigms.
Revised: Such a framework enables researchers to gain a better understanding of the progress and limitations of current DOM methods, which in turn can provide valuable insights for advancing DOM methods across all paradigms.

Original:  D A XB ENCH : D EFORMABLE - AND -J AX B ENCHMARK
Revised:  D A XB ENCH : D EFORMABLE - AND -J AX B ENCHMARK

Original: DaXBench is a general-purpose differentiable DOM simulation platform that covers a wide range of deformable objects and manipulation tasks.
Revised: DaXBench is a general-purpose differentiable DOM simulation platform that covers a wide range of deformable objects and manipulation tasks.

Original: To facilitate benchmarking of DOM methods from all paradigms, particularly differentiable methods, we developed our own differentiable simulator,
Revised: To facilitate benchmarking of DOM methods from all paradigms, particularly differentiable methods, we developed our own differentiable simulator,

Original: Deformable-and-JAX (DaX), which is both highly efficient and parallelizable.
Revised: Deformable-and-JAX (DaX), which is both highly efficient and parallelizable.

Original: In this section, we first provide a brief overview of our simulator, DaX, and then focus on introducing the benchmark tasks we have implemented for DaXBench.
Revised: In this section, we first provide a brief overview of our simulator, DaX, and then focus on introducing the benchmark tasks we have implemented for DaXBench.

Original: 3. D A X
Revised: 3. D A X

Original: DaX is a differentiable simulator designed to model various types of deformable object manipulation with high parallelization capabilities.
Revised: DaX is a differentiable simulator designed to model various types of deformable object manipulation with high parallelization capabilities.

Original: A comparison with other existing simulators can be found in Table 1. DaX is implemented using JAX (Bradbury et al., 2018), which supports highly parallelized computation with excellent automatic differentiation directly optimized at the GPU kernel level.
Revised: A comparison with other existing simulators can be found in Table 1. DaX is implemented using JAX (Bradbury et al., 2018), which supports highly parallelized computation with excellent automatic differentiation directly optimized at the GPU kernel level.

Original: Moreover, DaX can be seamlessly integrated with the numerous learning algorithms implemented using JAX, allowing for the entire computation graph to be highly parallelized end-to-end.
Revised: Moreover, DaX can be seamlessly integrated with the numerous learning algorithms implemented using JAX, allowing for the entire computation graph to be highly parallelized end-to-end.

Original: State Representations and Dynamic Models DaX adopts different state representations and dynamic systems to model different types of deformable objects with distinct underlying deformation and dynamics.
Revised: State Representations and Dynamic Models DaX adopts different state representations and dynamic systems to model different types of deformable objects with distinct underlying deformation and dynamics.

Original: To model liquid and rope, which undergo significant deformation, DaX represents their states using particles and uses Moving Least Square Material Point Method (MLS-MPM)
Revised: To model liquid and rope, which undergo significant deformation, DaX represents their states using particles and uses Moving Least Square Material Point Method (MLS-MPM)

Original: (Hu et al., 2018; Sulsky et al., 1994) to model their dynamics on the particle level.
Revised: (Hu et al., 2018; Sulsky et al., 1994) to model their dynamics on the particle level.

Original: For cloth, which only undergoes limited deformation, DaX models its state using mesh and uses a less computationally expensive mass-spring system to model its dynamics.
Revised: For cloth, which only undergoes limited deformation, DaX models its state using mesh and uses a less computationally expensive mass-spring system to model its dynamics.

Original: These modeling choices allow DaX to strike
Revised: These modeling choices allow DaX to strike

Original: [Figure]
Revised: [Figure]

Original: a balance between modeling capacity and complexity.
Revised: a balance between modeling capacity and complexity.

Original: For more information on the object models used in DaX please refer to Appendix section A.1.
Revised: For more information on the object models used in DaX please refer to Appendix section A.1.

Original: Designed for Customization DaX allows full customization of new tasks based on the existing framework.
Revised: Designed for Customization DaX allows full customization of new tasks based on the existing framework.

Original: Objects with arbitrary shapes and different rigidness can be added to the simulator with ease.
Revised: Objects with arbitrary shapes and different rigidness can be added to the simulator with ease.

Original: Moreover, primitive actions can also be customized.
Revised: Moreover, primitive actions can also be customized.

Original: For a code example of how to customize a new environment in DaX, please refer to Appendix section A.2.
Revised: For a code example of how to customize a new environment in DaX, please refer to Appendix section A.2.

Original: Saving Memory DaX uses two tricks to optimize the efficiency and memory consumption of the end-to-end gradient computation, namely lazy dynamic update and “checkpointing method” (Qiao et al., 2021; Chen et al., 2016; Griewank & Walther, 2000).
Revised: Saving Memory DaX uses two tricks to optimize the efficiency and memory consumption of the end-to-end gradient computation, namely lazy dynamic update and “checkpointing method” (Qiao et al., 2021; Chen et al., 2016; Griewank & Walther, 2000).

Original: To optimize the time and memory consumption of MPM for a single timestep, DaX lazily updates the dynamic of a small region affected by the manipulation at that step.
Revised: To optimize the time and memory consumption of MPM for a single timestep, DaX lazily updates the dynamic of a small region affected by the manipulation at that step.

Original: Moreover, DaX only stores the values of a few states (i.e., checkpoints) during the forward pass.
Revised: Moreover, DaX only stores the values of a few states (i.e., checkpoints) during the forward pass.

Original: During the backward propagation, DaX re-computes segments of the forward values starting from every stored state in reverse order sequentially for the overall gradient.
Revised: During the backward propagation, DaX re-computes segments of the forward values starting from every stored state in reverse order sequentially for the overall gradient.

Original: For more details on these techniques, please refer to Appendix section A.4.
Revised: For more details on these techniques, please refer to Appendix section A.4.

Original: 3. B ENCHMARK T ASKS
Revised: 3. B ENCHMARK T ASKS

Original: DaXBench provides a comprehensive set of Deformable Object Manipulation tasks, including liquid pouring, rope wiping, cloth folding, and sculpting elastoplastic objects, as illustrated in Figure 2.
Revised: DaXBench provides a comprehensive set of Deformable Object Manipulation tasks, including liquid pouring, rope wiping, cloth folding, and sculpting elastoplastic objects, as illustrated in Figure 2.

Original: These tasks are representative of the common deformable objects encountered in daily life.
Revised: These tasks are representative of the common deformable objects encountered in daily life.

Original: Additionally, we propose several new challenging long-horizon tasks that require finer-grained control, allowing for better benchmarking of the scalability of DOM methods.
Revised: Additionally, we propose several new challenging long-horizon tasks that require finer-grained control, allowing for better benchmarking of the scalability of DOM methods.

Original: The horizon of each task is determined by how many steps a human expert takes to finish the task.
Revised: The horizon of each task is determined by how many steps a human expert takes to finish the task.

Original: DaXBench provides a standardized platform for comparing existing methods and future work on DOM.
Revised: DaXBench provides a standardized platform for comparing existing methods and future work on DOM.

Original: In this section, we provide detailed information on the tasks supported by DaXBench.
Revised: In this section, we provide detailed information on the tasks supported by DaXBench.

Original: Long Horizon Tasks The time complexity for the search-based planning method grows exponentially with the planning horizon, known as the “curse of history”; this is exacerbated by the large state space of deformable objects.
Revised: Long Horizon Tasks The time complexity for the search-based planning method grows exponentially with the planning horizon, known as the “curse of history”; this is exacerbated by the large state space of deformable objects.

Original: However, the existing DOM benchmark tasks typically have relatively short horizons.
Revised: However, the existing DOM benchmark tasks typically have relatively short horizons.

Original: For example, the cloth folding and unfolding tasks in SoftGym consist of only 1-3 steps.
Revised: For example, the cloth folding and unfolding tasks in SoftGym consist of only 1-3 steps.

Original: DaXBench addresses this limitation by providing tasks with longer horizons, enabling better evaluation of the scalability of search-based planning methods.
Revised: DaXBench addresses this limitation by providing tasks with longer horizons, enabling better evaluation of the scalability of search-based planning methods.

Original: Tasks without Macro-Actions A macro-action can be seen as executing raw actions for multiple steps.
Revised: Tasks without Macro-Actions A macro-action can be seen as executing raw actions for multiple steps.

Original: For example, we can define a pick-and-place or push macro-action as a 6-dimensional real vector ( x, y, z, x ′ , y ′ , z ′ ) representing the start ( x, y, z ) and end ( x ′ , y ′ , z ′ ) positions of the gripper, and the raw actions are Cartesian velocities of the gripper.
Revised: For example, we can define a pick-and-place or push macro-action as a 6-dimensional real vector ( x, y, z, x ′ , y ′ , z ′ ) representing the start ( x, y, z ) and end ( x ′ , y ′ , z ′ ) positions of the gripper, and the raw actions are Cartesian velocities of the gripper.

Original: Macro-actions reduce the dimension of the action space and the effective horizon, and hence make scaling easier for the DOM methods.
Revised: Macro-actions reduce the dimension of the action space and the effective horizon, and hence make scaling easier for the DOM methods.

Original: However, macro-actions might be too coarse for some tasks.
Revised: However, macro-actions might be too coarse for some tasks.

Original: For example, when whipping a rope to hit a target location, the momentum needs to be adjusted at a relatively high frequency throughout the execution, making macro-actions unsuitable.
Revised: For example, when whipping a rope to hit a target location, the momentum needs to be adjusted at a relatively high frequency throughout the execution, making macro-actions unsuitable.

Original: DaXBench includes tasks that do not have suitable macro-actions to provide a more comprehensive evaluation of DOM methods.
Revised: DaXBench includes tasks that do not have suitable macro-actions to provide a more comprehensive evaluation of DOM methods.

Original: 3.2. L IQUID M ANIPULATION
Revised: 3.2. L IQUID M ANIPULATION

Original: Pour-Water Pour a bowl of water into the target bowl as quickly as possible.
Revised: Pour-Water Pour a bowl of water into the target bowl as quickly as possible.

Original: The agent directly controls the velocity and rotation (in Cartesian space) of the gripper that holds the bowl.
Revised: The agent directly controls the velocity and rotation (in Cartesian space) of the gripper that holds the bowl.

Original: In particular, the action space of the agent is a 4-dimensional vector a =
Revised: In particular, the action space of the agent is a 4-dimensional vector a =

Original: ( v x , v y , v z , w ) , for which ( v x , v y , v z ) represents the linear velocity of the gripper, and w denotes the angular velocity around the wrist.
Revised: ( v x , v y , v z , w ) , for which ( v x , v y , v z ) represents the linear velocity of the gripper, and w denotes the angular velocity around the wrist.

Original: Pour-Soup Pour a bowl of soup with various solid ingredients into the target bowl as fast as possible.
Revised: Pour-Soup Pour a bowl of soup with various solid ingredients into the target bowl as fast as possible.

Original: The actions are the same as those of Pour-Water.
Revised: The actions are the same as those of Pour-Water.

Original: This task is more challenging than Pour-Water due to the additional interactions between the liquid and the solid ingredients.
Revised: This task is more challenging than Pour-Water due to the additional interactions between the liquid and the solid ingredients.

Original: In particular, the agent needs to promptly react when a solid ingredient is transferred, so as to 1) adjust for the sudden decrease in load on the gripper and 2) control the momentum of the ingredient when hitting the soup surface to minimize spilling.
Revised: In particular, the agent needs to promptly react when a solid ingredient is transferred, so as to 1) adjust for the sudden decrease in load on the gripper and 2) control the momentum of the ingredient when hitting the soup surface to minimize spilling.

Original: Neither tasks have suitable high-level macro-actions and the task horizons are 100 steps
Revised: Neither tasks have suitable high-level macro-actions and the task horizons are 100 steps

Original: . 3.2. R OPE M ANIPULATION
Revised: . 3.2. R OPE M ANIPULATION

Original: Push-Rope Push the rope to the pre-specified configuration as quickly as possible.
Revised: Push-Rope Push the rope to the pre-specified configuration as quickly as possible.

Original: We randomly initialize the configuration of the rope.
Revised: We randomly initialize the configuration of the rope.

Original: The agent uses a single gripper to execute the push macroactions.
Revised: The agent uses a single gripper to execute the push macroactions.

Original: The horizon for this task is 6 steps.
Revised: The horizon for this task is 6 steps.

Original: Whip-Rope Whip the rope into a target configuration.
Revised: Whip-Rope Whip the rope into a target configuration.

Original: The agent holds one end of the rope and controls the gripper’s velocity to whip the rope.
Revised: The agent holds one end of the rope and controls the gripper’s velocity to whip the rope.

Original: In particular, the action space of this task is a 3dimensional vector a
Revised: In particular, the action space of this task is a 3dimensional vector a

Original: = ( v x , v y , v z ) that denotes the velocity of the gripper in the Cartesian space.
Revised: = ( v x , v y , v z ) that denotes the velocity of the gripper in the Cartesian space.

Original: This task cannot be completed by the abstract macro-actions since the momentum and the deformation of the rope has to be adjusted at a relatively high frequency.
Revised: This task cannot be completed by the abstract macro-actions since the momentum and the deformation of the rope has to be adjusted at a relatively high frequency.

Original: The task horizon is 70 steps.
Revised: The task horizon is 70 steps.

Original: 3.2. C LOTH M ANIPULATION
Revised: 3.2. C LOTH M ANIPULATION

Original: Fold-Cloth Fold a piece of flattened cloth and move it to a target location.
Revised: Fold-Cloth Fold a piece of flattened cloth and move it to a target location.

Original: This task has two variants: the easy version requires the agent to execute 1 fold and the difficult version 3 folds.
Revised: This task has two variants: the easy version requires the agent to execute 1 fold and the difficult version 3 folds.

Original: For both variants, the task begins with the cloth lying flat on the table at an arbitrary position.
Revised: For both variants, the task begins with the cloth lying flat on the table at an arbitrary position.

Original: The agent executes pick-and-place macro-actions.
Revised: The agent executes pick-and-place macro-actions.

Original: The task horizon is 3 and 4 for the easy and difficult versions respectively.
Revised: The task horizon is 3 and 4 for the easy and difficult versions respectively.

Original: Fold-T-shirt Fold a T-shirt to a target location.
Revised: Fold-T-shirt Fold a T-shirt to a target location.

Original: The task begins with the T-shirt lying flat on the table at an arbitrary position.
Revised: The task begins with the T-shirt lying flat on the table at an arbitrary position.

Original: The agent executes pick-and-place macro-actions.
Revised: The agent executes pick-and-place macro-actions.

Original: The task horizon is 4.
Revised: The task horizon is 4.

Original: Unfold-Cloth Flatten a piece of folded cloth to a target location as fast as possible.
Revised: Unfold-Cloth Flatten a piece of folded cloth to a target location as fast as possible.

Original: This task also has two variants that differ in the initial configurations of the folded cloth: the cloth in the easy version has been folded once and that in the difficult version has been folded three times.
Revised: This task also has two variants that differ in the initial configurations of the folded cloth: the cloth in the easy version has been folded once and that in the difficult version has been folded three times.

Original: This task has the same action space as the Fold-Cloth task.
Revised: This task has the same action space as the Fold-Cloth task.

Original: The task horizon is 10 steps for both variants.
Revised: The task horizon is 10 steps for both variants.

Original: All of these tasks share similar APIs to the OpenAI Gym’s standard environments.
Revised: All of these tasks share similar APIs to the OpenAI Gym’s standard environments.

Original: We provide a general template to define tasks in our simulator.
Revised: We provide a general template to define tasks in our simulator.

Original: This not only better organizes the existing task environments but also allows the users to customize the existing task environments to better suit their research requirements.
Revised: This not only better organizes the existing task environments but also allows the users to customize the existing task environments to better suit their research requirements.

Original: The variables and constants defined in the template are easily interpretable quantities that correspond to real physical semantics.
Revised: The variables and constants defined in the template are easily interpretable quantities that correspond to real physical semantics.

Original: This allows the users to better understand the environment, which can also help with the development and debugging process.
Revised: This allows the users to better understand the environment, which can also help with the development and debugging process.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we aim to answer the following two questions: 1) Are the existing representative methods capable of solving the tasks in DaXBench? 2)
Revised: In this section, we aim to answer the following two questions: 1) Are the existing representative methods capable of solving the tasks in DaXBench? 2)

Original: Within each paradigm, will the differentiable simulator help in completing the tasks?
Revised: Within each paradigm, will the differentiable simulator help in completing the tasks?

Original: To investigate these two questions, we benchmark eight representative methods of deformable object manipulation, covering planning, imitation learning, and reinforcement learning.
Revised: To investigate these two questions, we benchmark eight representative methods of deformable object manipulation, covering planning, imitation learning, and reinforcement learning.

Original: Each paradigm has different assumptions and requirements for input knowledge.
Revised: Each paradigm has different assumptions and requirements for input knowledge.

Original: We summarize the baseline methods and the relevant information in Table 2.
Revised: We summarize the baseline methods and the relevant information in Table 2.

Original: [Figure]
Revised: [Figure]

Original: Reinforcement
Revised: Reinforcement

Original: [Figure]
Revised: [Figure]

Original: Imitation
Revised: Imitation

Original: [Figure]
Revised: [Figure]

Original: Planning
Revised: Planning

Original: Methods
Revised: Methods

Original: PPO
Revised: PPO

Original: SHAC
Revised: SHAC

Original: APG
Revised: APG

Original: [Figure]
Revised: [Figure]

Original: ILD
Revised: ILD

Original: CEM
Revised: CEM

Original: [Figure]
Revised: [Figure]

Original: MPC diff
Revised: MPC diff

Original: [Figure]
Revised: [Figure]

Original: diff
Revised: diff

Original: [Figure]
Revised: [Figure]

Original: # expert demos — — —  — — —
Revised: # expert demos — — —  — — —

Original: Reward
Revised: Reward

Original: [Figure]
Revised: [Figure]

Original: ✓ ✓ ✓ — — ✓ ✓ ✓
Revised: ✓ ✓ ✓ — — ✓ ✓ ✓

Original: Differentiability ✗ ✓ ✓ ✗ ✓ ✗ ✓ ✓
Revised: Differentiability ✗ ✓ ✓ ✗ ✓ ✗ ✓ ✓

Original: [Figure]
Revised: [Figure]

Original: , with larger values being closer to the goal state.
Revised: , with larger values being closer to the goal state.

Original: The x-axis is the number of training iterations.
Revised: The x-axis is the number of training iterations.

Original: All methods are trained in the same number of batched environments.
Revised: All methods are trained in the same number of batched environments.

Original: We report the mean and variance of the performance for 5 seeds and each seed has 20 rollouts with different initializations.
Revised: We report the mean and variance of the performance for 5 seeds and each seed has 20 rollouts with different initializations.

Original: We refer the readers to Table 4 for the numerical results in the appendix.
Revised: We refer the readers to Table 4 for the numerical results in the appendix.

Original: 4. E XPERIMENTAL S ETUP
Revised: 4. E XPERIMENTAL S ETUP

Original: Reward function For each task, the goal is specified by the desired final positions of the set of particles, g , representing the deformable object.
Revised: Reward function For each task, the goal is specified by the desired final positions of the set of particles, g , representing the deformable object.

Original: In this setup, the reward can be intuitively defined by how well the current object particles match with those of the goal.
Revised: In this setup, the reward can be intuitively defined by how well the current object particles match with those of the goal.

Original: Hence, we define the groundtruth reward as r gt ( s, a ) =
Revised: Hence, we define the groundtruth reward as r gt ( s, a ) =

Original: exp( − λD ( s ′ , g )) , where s ′ is the next state resulted from a at the current state s , and D ( s, g ) is a non-negative distance measure between the positions of the object’s particles s and those of the goal g .
Revised: exp( − λD ( s ′ , g )) , where s ′ is the next state resulted from a at the current state s , and D ( s, g ) is a non-negative distance measure between the positions of the object’s particles s and those of the goal g .

Original: However, using the ground-truth reward alone is inefficient for learning.
Revised: However, using the ground-truth reward alone is inefficient for learning.

Original: The robot’s end-effector can only change the current object pose by contacting it.
Revised: The robot’s end-effector can only change the current object pose by contacting it.

Original: Therefore, to most efficiently manipulate the object to the desired configuration, we add an auxiliary reward to encourage the robot to contact the object.
Revised: Therefore, to most efficiently manipulate the object to the desired configuration, we add an auxiliary reward to encourage the robot to contact the object.

Original: We define the auxiliary reward as r aux
Revised: We define the auxiliary reward as r aux

Original: ( s, a ) =
Revised: ( s, a ) =

Original: exp( − L 2 ( s, a )) , where L 2 ( s, a ) measures the
Revised: exp( − L 2 ( s, a )) , where L 2 ( s, a ) measures the

Original: L 2 distance between the end-effector and the object.
Revised: L 2 distance between the end-effector and the object.

Original: During training, we use the sum of the groundtruth and auxiliary reward functions r gt + r aux , and during evaluation, we only use the ground-truth reward r gt .
Revised: During training, we use the sum of the groundtruth and auxiliary reward functions r gt + r aux , and during evaluation, we only use the ground-truth reward r gt .

Original: 4. R EINFORCEMENT L EARNING
Revised: 4. R EINFORCEMENT L EARNING

Original: Methods.
Revised: Methods.

Original: We benchmark PPO (Schulman et al., 2017), SHAC (Xu et al., 2022), and APG (Freeman et al., 2021) as the representative RL methods.
Revised: We benchmark PPO (Schulman et al., 2017), SHAC (Xu et al., 2022), and APG (Freeman et al., 2021) as the representative RL methods.

Original: PPO is a widely used model-free nondifferentiable RL method with competitive performance, while SHAC and APG are the two latest
Revised: PPO is a widely used model-free nondifferentiable RL method with competitive performance, while SHAC and APG are the two latest

Original: RL methods that utilize the differentiable simulator.
Revised: RL methods that utilize the differentiable simulator.

Original: The computation graph of SHAC is illustrated in Figure 4a.
Revised: The computation graph of SHAC is illustrated in Figure 4a.

Original: By comparing the performance of the DaXBench tasks across these three methods, we aim to further study the benefits and limitations of applying the differentiable physics to RL.
Revised: By comparing the performance of the DaXBench tasks across these three methods, we aim to further study the benefits and limitations of applying the differentiable physics to RL.

Original: Overall Performance.
Revised: Overall Performance.

Original: The learning curves of RL methods are shown in Figure 3.
Revised: The learning curves of RL methods are shown in Figure 3.

Original: The performance difference between PPO and the differentiable RL methods shows opposite trends in the high-level macro-action-based and local-level control tasks.
Revised: The performance difference between PPO and the differentiable RL methods shows opposite trends in the high-level macro-action-based and local-level control tasks.

Original: To our surprise, for most of the short-horizon
Revised: To our surprise, for most of the short-horizon

Original: [Table]
Revised: [Table]

Original: macro-action-based tasks
Revised: macro-action-based tasks

Original: , PPO performs consistently better than the differentiable RL methods.
Revised: , PPO performs consistently better than the differentiable RL methods.

Original: This seems to contradict the experiment results of SHAC (Xu et al., 2022), which shows stronger performance than PPO in its own simulator on a wide range of context-rich MuJoCo tasks.
Revised: This seems to contradict the experiment results of SHAC (Xu et al., 2022), which shows stronger performance than PPO in its own simulator on a wide range of context-rich MuJoCo tasks.

Original: However, for long-horizon tasks with low-level controls, the differentiable RL methods, especially APG, outperform PPO by a large margin.
Revised: However, for long-horizon tasks with low-level controls, the differentiable RL methods, especially APG, outperform PPO by a large margin.

Original: Main Challenge for the differentiable-physics-based RL methods: Exploration.
Revised: Main Challenge for the differentiable-physics-based RL methods: Exploration.

Original: We argue that, on high-level macro-action-based tasks, the main limiting factor of the performance of differentiable-physics-based RL is the lack of exploration during training.
Revised: We argue that, on high-level macro-action-based tasks, the main limiting factor of the performance of differentiable-physics-based RL is the lack of exploration during training.

Original: The need to balance this trade-off is exaggerated by the enormous DoFs and the complex dynamics unique to the DOM tasks.
Revised: The need to balance this trade-off is exaggerated by the enormous DoFs and the complex dynamics unique to the DOM tasks.

Original: Especially for the high-level macro-action-based tasks, their non-smooth/discontinuous/nonconvex optimization landscape necessitates using a good exploration strategy.
Revised: Especially for the high-level macro-action-based tasks, their non-smooth/discontinuous/nonconvex optimization landscape necessitates using a good exploration strategy.

Original: However, the existing differentiable-physics-based RL methods rely exclusively on the simulation gradient to optimize the policy.
Revised: However, the existing differentiable-physics-based RL methods rely exclusively on the simulation gradient to optimize the policy.

Original: They are not entropy regularized; hence, they may quickly collapse to the local optimal.
Revised: They are not entropy regularized; hence, they may quickly collapse to the local optimal.

Original: Without explicit exploration, the differentiable-physics-based RL methods fail to reach the near-optimal state using the simulation gradients alone.
Revised: Without explicit exploration, the differentiable-physics-based RL methods fail to reach the near-optimal state using the simulation gradients alone.

Original: Taking Fold-Cloth-3 as an example, if the randomly initialized policy never touches the cloth, the local simulation gradients cannot guide the agent to touch the cloth since the rewards and, consequently, the gradients for all not-in-contact actions are zero.
Revised: Taking Fold-Cloth-3 as an example, if the randomly initialized policy never touches the cloth, the local simulation gradients cannot guide the agent to touch the cloth since the rewards and, consequently, the gradients for all not-in-contact actions are zero.

Original: Differentiable-physics-based RL are sensitive to the Optimization Landscape.
Revised: Differentiable-physics-based RL are sensitive to the Optimization Landscape.

Original: Differentiablephysics-based RL can largely improve the performance of some low-level control tasks if the optimization landscape is smooth and convex.
Revised: Differentiablephysics-based RL can largely improve the performance of some low-level control tasks if the optimization landscape is smooth and convex.

Original: Taking Whip-Rope as an example, the optimal low-level control sequence is a smooth motion trajectory that lies on a relatively smooth and well-behaved plane for the gradient-based methods.
Revised: Taking Whip-Rope as an example, the optimal low-level control sequence is a smooth motion trajectory that lies on a relatively smooth and well-behaved plane for the gradient-based methods.

Original: This explains why APG outperforms the non-differentiablephysics-based PPO by a large margin.
Revised: This explains why APG outperforms the non-differentiablephysics-based PPO by a large margin.

Original: 4. I MITATION L EARNING
Revised: 4. I MITATION L EARNING

Original: Methods.
Revised: Methods.

Original: We benchmark Transporter (Seita et al., 2021) and ILD (Chen et al., 2022) as the two representative Imitation Learning DOM methods.
Revised: We benchmark Transporter (Seita et al., 2021) and ILD (Chen et al., 2022) as the two representative Imitation Learning DOM methods.

Original: Transporter is a popular DOM method that is well-known for its simple implementation and competitive performance.
Revised: Transporter is a popular DOM method that is well-known for its simple implementation and competitive performance.

Original: We use particles as input to evaluate the performance of Transporter in our experiment for a fair comparison.
Revised: We use particles as input to evaluate the performance of Transporter in our experiment for a fair comparison.

Original: The original implementation of Transporter uses RGBD as input.
Revised: The original implementation of Transporter uses RGBD as input.

Original: We refer the readers to the Appendix section D for the performance of the Transporter with RGB channels.
Revised: We refer the readers to the Appendix section D for the performance of the Transporter with RGB channels.

Original: ILD is the latest differentiablephysics-based IL method; it utilizes the simulation gradients to reduce the required number of expert demonstrations and to improve performance.
Revised: ILD is the latest differentiablephysics-based IL method; it utilizes the simulation gradients to reduce the required number of expert demonstrations and to improve performance.

Original: Note that since Transporter abstracts its action space to pick-and-place actions to simplify the learning, it cannot be extended to the low-level control tasks.
Revised: Note that since Transporter abstracts its action space to pick-and-place actions to simplify the learning, it cannot be extended to the low-level control tasks.

Original: Our comparison of the IL methods will first focus on the high-level macro-action-based tasks, then we will analyze the performance of ILD with the expert.
Revised: Our comparison of the IL methods will first focus on the high-level macro-action-based tasks, then we will analyze the performance of ILD with the expert.

Original: Overall performance For the high-level macro-action-based tasks, ILD outperforms Transporter by a large margin for all tasks.
Revised: Overall performance For the high-level macro-action-based tasks, ILD outperforms Transporter by a large margin for all tasks.

Original: For example, ILD outperforms Transporter in Fold-Cloth-1, Fold-Cloth3, and Unfold-Cloth-1.
Revised: For example, ILD outperforms Transporter in Fold-Cloth-1, Fold-Cloth3, and Unfold-Cloth-1.

Original: We highlight that this significant performance gain is achieved by using a much smaller number of expert demonstrations.
Revised: We highlight that this significant performance gain is achieved by using a much smaller number of expert demonstrations.

Original: We attribute this improvement to the additional information provided by the simulation gradients, where now ILD can reason over the ground-truth
Revised: We attribute this improvement to the additional information provided by the simulation gradients, where now ILD can reason over the ground-truth

Original: [Figure]
Revised: [Figure]

Original: dynamics to bring the trajectory closer to the expert, at least locally.
Revised: dynamics to bring the trajectory closer to the expert, at least locally.

Original: As illustrated in Figure 4, the gradients used to optimize each action comes from the globally optimal expert demonstrations 1) at the current step and 2) in all the steps after.
Revised: As illustrated in Figure 4, the gradients used to optimize each action comes from the globally optimal expert demonstrations 1) at the current step and 2) in all the steps after.

Original: In contrast, the gradients used in SHAC only count the reward at the current step.
Revised: In contrast, the gradients used in SHAC only count the reward at the current step.

Original: The differentiable RL faces the challenge of exploding/vanishing gradients and lack of exploration.
Revised: The differentiable RL faces the challenge of exploding/vanishing gradients and lack of exploration.

Original: Differentiable IL alleviates these problems by having step-wise, globally optimal guidance from expert demonstrations.
Revised: Differentiable IL alleviates these problems by having step-wise, globally optimal guidance from expert demonstrations.

Original: Challenge for ILD: Unbalanced State Representations.
Revised: Challenge for ILD: Unbalanced State Representations.

Original: ILD does not perform well on tasks like
Revised: ILD does not perform well on tasks like

Original: Pour-Water and Pour-Soup with unbalanced state representations, for example, 1000 × 3 features of particles and 6 gripper states.
Revised: Pour-Water and Pour-Soup with unbalanced state representations, for example, 1000 × 3 features of particles and 6 gripper states.

Original: The learned signals from the particles overwhelm the signals from the gripper states.
Revised: The learned signals from the particles overwhelm the signals from the gripper states.

Original: Therefore, ILD cannot learn useful strategies with unbalanced learning signals.
Revised: Therefore, ILD cannot learn useful strategies with unbalanced learning signals.

Original: Softgym (Lin et al., 2021) also concurs with the results that policies with reduced state representation perform better than those with the full state.
Revised: Softgym (Lin et al., 2021) also concurs with the results that policies with reduced state representation perform better than those with the full state.

Original: A more representative representation of the states needs to be considered and we leave it for future study.
Revised: A more representative representation of the states needs to be considered and we leave it for future study.

Original: In conclusion, differentiable methods can optimize the policies efficiently if the optimization landscape is well-behaved.
Revised: In conclusion, differentiable methods can optimize the policies efficiently if the optimization landscape is well-behaved.

Original: For tasks that require tremendous explorations or with sparse rewards, gradient-based methods suffer severely from the local minima and gradient instability issues.
Revised: For tasks that require tremendous explorations or with sparse rewards, gradient-based methods suffer severely from the local minima and gradient instability issues.

Original: 4. P LANNING
Revised: 4. P LANNING

Original: Methods.
Revised: Methods.

Original: We benchmark the classic CEM-MPC (Richards, 2005) as a representative random shooting method that uses a gradient-free method to optimize the action sequences for the highest final reward.
Revised: We benchmark the classic CEM-MPC (Richards, 2005) as a representative random shooting method that uses a gradient-free method to optimize the action sequences for the highest final reward.

Original: To study the effect of using differentiable physics, we implement two versions of differentiable MPC.
Revised: To study the effect of using differentiable physics, we implement two versions of differentiable MPC.

Original: The basic computation graph for both is illustrated in Figure 4c.
Revised: The basic computation graph for both is illustrated in Figure 4c.

Original: In particular, both differentiable MPC baselines maximize the immediate reward of each action using differentiable physics.
Revised: In particular, both differentiable MPC baselines maximize the immediate reward of each action using differentiable physics.

Original: The difference lies in the initial action sequences: diff-MPC is initialized with a random action sequence, while diff-CEM-MPC with an action sequence optimized by CEM-MPC.
Revised: The difference lies in the initial action sequences: diff-MPC is initialized with a random action sequence, while diff-CEM-MPC with an action sequence optimized by CEM-MPC.

Original: Overall Performance.
Revised: Overall Performance.

Original: The diff-CEM-MPC performs consistently better than or on par with CEMMPC, while Diff-MPC fails to plan any reasonable control sequences for most of the tasks.
Revised: The diff-CEM-MPC performs consistently better than or on par with CEMMPC, while Diff-MPC fails to plan any reasonable control sequences for most of the tasks.

Original: Simulation Gradients as Additional Optimization Signals.
Revised: Simulation Gradients as Additional Optimization Signals.

Original: The vastly different performance between the two versions of the differentiable baselines brings us insights into how differentiable physics helps in planning.
Revised: The vastly different performance between the two versions of the differentiable baselines brings us insights into how differentiable physics helps in planning.

Original: Simulation gradients provide additional signals in the direction to optimize the actions, thereby reducing the sample complexity.
Revised: Simulation gradients provide additional signals in the direction to optimize the actions, thereby reducing the sample complexity.

Original: However, similar to any gradient-based optimization methods, differentiable planning methods are sensitive to the non-convex/non-smooth optimization landscape and face a major limitation of being stuck at the local optimal solution.
Revised: However, similar to any gradient-based optimization methods, differentiable planning methods are sensitive to the non-convex/non-smooth optimization landscape and face a major limitation of being stuck at the local optimal solution.

Original: DOM’s enormous DoFs of DOM and complex dynamics exacerbate these challenges for the differentiable DOM planning methods.
Revised: DOM’s enormous DoFs of DOM and complex dynamics exacerbate these challenges for the differentiable DOM planning methods.

Original: Differentiable Planning needs Good Initialization.
Revised: Differentiable Planning needs Good Initialization.

Original: We attribute the poor performance of DiffMPC to its random control sequence initialization.
Revised: We attribute the poor performance of DiffMPC to its random control sequence initialization.

Original: In particular, this control sequence may significantly deviate from the optimal one over the non-smooth/non-convex landscape, such that the simulation gradients alone cannot bring the control sequence back to the optimal.
Revised: In particular, this control sequence may significantly deviate from the optimal one over the non-smooth/non-convex landscape, such that the simulation gradients alone cannot bring the control sequence back to the optimal.

Original: In contrast, DiffCEM-MPC is initialized with the optimized control sequence by CEM-MPC, which relies on iterative sampling followed by elite selection to escape the local optimality.
Revised: In contrast, DiffCEM-MPC is initialized with the optimized control sequence by CEM-MPC, which relies on iterative sampling followed by elite selection to escape the local optimality.

Original: Hence, the control sequence fed into the differentiable step is already sufficiently close to the globally optimal solution.
Revised: Hence, the control sequence fed into the differentiable step is already sufficiently close to the globally optimal solution.

Original: The
Revised: The

Original: [Figure]
Revised: [Figure]

Original: simulation gradients help diff-CEM-MPC to optimize the action sequences further locally, which explains the performance gain.
Revised: simulation gradients help diff-CEM-MPC to optimize the action sequences further locally, which explains the performance gain.

Original: To summarize, the differentiable simulator can further optimize the near-optimal control sequences locally with much low sample complexity.
Revised: To summarize, the differentiable simulator can further optimize the near-optimal control sequences locally with much low sample complexity.

Original: Yet, its performance critically relies on a good initialization and a locally smooth optimization landscape.
Revised: Yet, its performance critically relies on a good initialization and a locally smooth optimization landscape.

Original: 4. S IM -2-R EAL G AP
Revised: 4. S IM -2-R EAL G AP

Original: Granted that the physical engine of DaX is built upon a state-of-the-art simulation method, the fidelity and correctness still need to be tested.
Revised: Granted that the physical engine of DaX is built upon a state-of-the-art simulation method, the fidelity and correctness still need to be tested.

Original: To testify to the correctness of our simulated dynamics to real-world physics, we carry out a real robot experiment on a Kinova Gen 3 robot.
Revised: To testify to the correctness of our simulated dynamics to real-world physics, we carry out a real robot experiment on a Kinova Gen 3 robot.

Original: We deploy
Revised: We deploy

Original: CEM-MPC with DaX as the predictor model to a Push-Rope task, as shown in Figure 5.
Revised: CEM-MPC with DaX as the predictor model to a Push-Rope task, as shown in Figure 5.

Original: Our study finds that the resultant state trajectories are similar in simulation and in reality.
Revised: Our study finds that the resultant state trajectories are similar in simulation and in reality.

Original: This validates the fidelity of the tasks implemented based on our engine, and the performance of the DOM methods in our simulator provides hints about their performance on the real robot.
Revised: This validates the fidelity of the tasks implemented based on our engine, and the performance of the DOM methods in our simulator provides hints about their performance on the real robot.

Original: We refer the reader to the appendix for details.
Revised: We refer the reader to the appendix for details.

Original: In addition, the differentiability of DaX enables system identification that can reduce the sim-2-real gap (Le Lidec et al., 2021; Murthy et al., 2021).
Revised: In addition, the differentiability of DaX enables system identification that can reduce the sim-2-real gap (Le Lidec et al., 2021; Murthy et al., 2021).

Original: We can get an error term by comparing the observations in simulation and in reality; by differentiating this error with respect to the simulator parameters, we get gradients for parameter tuning.
Revised: We can get an error term by comparing the observations in simulation and in reality; by differentiating this error with respect to the simulator parameters, we get gradients for parameter tuning.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We introduced DaXBench, a general-purpose differentiable simulation framework for various deformable object manipulation tasks.
Revised: We introduced DaXBench, a general-purpose differentiable simulation framework for various deformable object manipulation tasks.

Original: To our knowledge, DaXBench is the first platform that simulates liquid, rope, fabric, and elastoplastic materials while being differentiable and highly parallelizable.
Revised: To our knowledge, DaXBench is the first platform that simulates liquid, rope, fabric, and elastoplastic materials while being differentiable and highly parallelizable.

Original: In addition to common DOM tasks, we also proposed several novel tasks, including
Revised: In addition to common DOM tasks, we also proposed several novel tasks, including

Original: WhipRope and PourSoup, to study the performance of DOM methods on long-horizon tasks with low-level controls.
Revised: WhipRope and PourSoup, to study the performance of DOM methods on long-horizon tasks with low-level controls.

Original: Our task coverage provides a comprehensive platform to compare existing DOM methods from all paradigms, and our empirical study sheds light on the benefits and limitations of differentiable-physics-based methods in DOM.
Revised: Our task coverage provides a comprehensive platform to compare existing DOM methods from all paradigms, and our empirical study sheds light on the benefits and limitations of differentiable-physics-based methods in DOM.

Original: We believe that DaXBenchcan assist in the algorithmic development of DOM methods, particularly those based on differentiable physics, across all paradigms.
Revised: We believe that DaXBenchcan assist in the algorithmic development of DOM methods, particularly those based on differentiable physics, across all paradigms.

Original: A CKNOWLEDGEMENTS
Revised: A CKNOWLEDGEMENTS

Original: This research is supported by the National Research Foundation, Singapore under its AI Singapore Programme (AISG Award
Revised: This research is supported by the National Research Foundation, Singapore under its AI Singapore Programme (AISG Award

Original: No: AISG2-PhD-2021-08-015T, AISG2-PhD-2022-01-036T, and
Revised: No: AISG2-PhD-2021-08-015T, AISG2-PhD-2022-01-036T, and

Original: AISG2-PhD-2021-08-014) and A ∗ STAR under the National Robotics Program (Grant No. 19200054).
Revised: AISG2-PhD-2021-08-014) and A ∗ STAR under the National Robotics Program (Grant No. 19200054).

Original: E THICS S TATEMENT
Revised: E THICS S TATEMENT

Original: DaXBench provides a benchmark for deformable object manipulation (DOM).
Revised: DaXBench provides a benchmark for deformable object manipulation (DOM).

Original: We believe that
Revised: We believe that

Original: DaXBench and its simulator DaX have no potential negative societal impacts.
Revised: DaXBench and its simulator DaX have no potential negative societal impacts.

Original: In addition, our work is on benchmarking the DOM methods in simulator, so we do not collect any data or conduct experiments with human subjects.
Revised: In addition, our work is on benchmarking the DOM methods in simulator, so we do not collect any data or conduct experiments with human subjects.

Original: In summary, we have read the ICLR Code of Ethics and ensured that our work conforms to them
Revised: In summary, we have read the ICLR Code of Ethics and ensured that our work conforms to them

Original: R EPRODUCIBILITY S TATEMENT
Revised: R EPRODUCIBILITY S TATEMENT

Original: In this paper imitation learning and planning experiments are averaged over 20 random rollouts and
Revised: In this paper imitation learning and planning experiments are averaged over 20 random rollouts and

Original: RL approaches use 100 rollouts.
Revised: RL approaches use 100 rollouts.

Original: We have included our source code as an easy-to-install package in the supplementary material.
Revised: We have included our source code as an easy-to-install package in the supplementary material.

Original: A I MPLEMENTATION D ETAILS A. O BJECTS
Revised: A I MPLEMENTATION D ETAILS A. O BJECTS

Original: DaXBench models a variety of deformable objects of different dimensionalities, ranging from 0dimensional liquid, 1-dimensional rope, to 2-dimensional cloth.
Revised: DaXBench models a variety of deformable objects of different dimensionalities, ranging from 0dimensional liquid, 1-dimensional rope, to 2-dimensional cloth.

Original: The underlying deformation and the dynamics vary distinctly for each dimension.
Revised: The underlying deformation and the dynamics vary distinctly for each dimension.

Original: In general, the lower the dimension is, the higher the degrees of freedom (DoFs) of the system, thus the more complex modeling that system becomes.
Revised: In general, the lower the dimension is, the higher the degrees of freedom (DoFs) of the system, thus the more complex modeling that system becomes.

Original: We use two different representations to model the deformable objects to balance the modeling capacity with its complexity: water and rope are modeled using Material Point Method (MPM), while the cloth is modeled using a mass-spring system.
Revised: We use two different representations to model the deformable objects to balance the modeling capacity with its complexity: water and rope are modeled using Material Point Method (MPM), while the cloth is modeled using a mass-spring system.

Original: Modelling Water and Rope using MPM Material Point Method (MPM) demonstrates strong modeling capacity and flexibility.
Revised: Modelling Water and Rope using MPM Material Point Method (MPM) demonstrates strong modeling capacity and flexibility.

Original: MPM models the deformable objects as a set of particles; each particle is represented by its position and some properties such as mass.
Revised: MPM models the deformable objects as a set of particles; each particle is represented by its position and some properties such as mass.

Original: The dynamics and the deformation of the object are computed by simulating the interactions among all individual particles, conforming to some pre-defined physics rules.
Revised: The dynamics and the deformation of the object are computed by simulating the interactions among all individual particles, conforming to some pre-defined physics rules.

Original: This Particle-level representation and the dynamics simulation allow MPM to model the object under severe deformation.
Revised: This Particle-level representation and the dynamics simulation allow MPM to model the object under severe deformation.

Original: We, therefore, choose to use MPM to model water and rope that can undergo severe deformation, for they can exploit the model capacity of MPM to its full potential.
Revised: We, therefore, choose to use MPM to model water and rope that can undergo severe deformation, for they can exploit the model capacity of MPM to its full potential.

Original: Modelling Cloth using Mass-Spring System Cloth only undergoes limited deformation as it has to conform to the canonical shape of the initial cloth through manipulation.
Revised: Modelling Cloth using Mass-Spring System Cloth only undergoes limited deformation as it has to conform to the canonical shape of the initial cloth through manipulation.

Original: This limited deformable does not require the strong modeling capacity of MPM, which is computationally expensive.
Revised: This limited deformable does not require the strong modeling capacity of MPM, which is computationally expensive.

Original: Therefore, rather than modeling the deformation on the Particle-level, we choose to model the cloth using the mass-spring system, which is a simpler model that uses the physical constraints as a prior.
Revised: Therefore, rather than modeling the deformation on the Particle-level, we choose to model the cloth using the mass-spring system, which is a simpler model that uses the physical constraints as a prior.

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: 1 import numpy as np 2 from
Revised: 1 import numpy as np 2 from

Original: .envs.registration import env_functions
Revised: .envs.registration import env_functions

Original: .action_size)) 8 obs, reward, done, info = env.step_diff(actions, state)
Revised: .action_size)) 8 obs, reward, done, info = env.step_diff(actions, state)

Original: Listing 1: Example of Running Cloth Environment 1 import jax 2 import numpy as np 3 from core.envs.registration import env_functions 4 5 env = env_functions["fold_cloth3"](batch_size=32, aux_reward=True) 6 obs, state = env.reset(env.simulator.key_global) 7 actions =
Revised: Listing 1: Example of Running Cloth Environment 1 import jax 2 import numpy as np 3 from core.envs.registration import env_functions 4 5 env = env_functions["fold_cloth3"](batch_size=32, aux_reward=True) 6 obs, state = env.reset(env.simulator.key_global) 7 actions =

Original: np.random.uniform(0, 1, (env.batch_size, env.action_size)) 8 9 @jax.jit 10 @jax.grad 11 def compute_grad(actions, state): obs, reward, done, info = env.step_diff(actions, state) return -reward.sum() 14 15 print("action gradients", compute_grad(actions, state))
Revised: np.random.uniform(0, 1, (env.batch_size, env.action_size)) 8 9 @jax.jit 10 @jax.grad 11 def compute_grad(actions, state): obs, reward, done, info = env.step_diff(actions, state) return -reward.sum() 14 15 print("action gradients", compute_grad(actions, state))

Original: Listing 2: Example of Computing Action Gradients via DaX
Revised: Listing 2: Example of Computing Action Gradients via DaX

Original: A. J AX TO PARALLELIZE
Revised: A. J AX TO PARALLELIZE

Original: One major limitation of the existing simulators is the lack of support for highly paralleled sampling.
Revised: One major limitation of the existing simulators is the lack of support for highly paralleled sampling.

Original: The key challenges for Deformable Object Manipulation are the complex governing equations and
Revised: The key challenges for Deformable Object Manipulation are the complex governing equations and

Original: [Figure]
Revised: [Figure]

Original: the enormous state space.
Revised: the enormous state space.

Original: The complex governing equations invalidate most of the analytical approaches since solving the complex governing equations is intractable.
Revised: The complex governing equations invalidate most of the analytical approaches since solving the complex governing equations is intractable.

Original: On the other hand, the success of the sampling-based methods hinges on the reasonable coverage of the state space.
Revised: On the other hand, the success of the sampling-based methods hinges on the reasonable coverage of the state space.

Original: The lack of support for highly paralleled sampling to better cover the state space is a major impediment to the works on Deformable Object Manipulation.
Revised: The lack of support for highly paralleled sampling to better cover the state space is a major impediment to the works on Deformable Object Manipulation.

Original: DaXBench fills in this gap by implementing its simulator using JAX, which supports highly paralleled sampling with great automatic differentiation.
Revised: DaXBench fills in this gap by implementing its simulator using JAX, which supports highly paralleled sampling with great automatic differentiation.

Original: Using our implementation, we can finish an iteration (both forward and backward pass) for 128 rollouts with 80 timesteps on a server with 4 2080-Ti GPUs in 3 seconds.
Revised: Using our implementation, we can finish an iteration (both forward and backward pass) for 128 rollouts with 80 timesteps on a server with 4 2080-Ti GPUs in 3 seconds.

Original: A. S AVING M EMORY
Revised: A. S AVING M EMORY

Original: Lazy Dynamic Update DaXBench optimizes the time and memory consumption of MPM by only updating the dynamic of a small region affected by the manipulation at each step.
Revised: Lazy Dynamic Update DaXBench optimizes the time and memory consumption of MPM by only updating the dynamic of a small region affected by the manipulation at each step.

Original: MPM models and updates the state and dynamics on the Particle-level; its time and memory consumption is proportional to the size of the environment, which can be arbitrarily large.
Revised: MPM models and updates the state and dynamics on the Particle-level; its time and memory consumption is proportional to the size of the environment, which can be arbitrarily large.

Original: Our insight is that, though the deformable object can potentially interact with the entire environment, only a small region will be affected at each time.
Revised: Our insight is that, though the deformable object can potentially interact with the entire environment, only a small region will be affected at each time.

Original: This inspires DaXBench to optimize the time and memory consumption of running MPM by exclusively considering the active region during the interactions.
Revised: This inspires DaXBench to optimize the time and memory consumption of running MPM by exclusively considering the active region during the interactions.

Original: Using this trick, DaXBench effectively reduces the computation time by two orders of magnitude on average.
Revised: Using this trick, DaXBench effectively reduces the computation time by two orders of magnitude on average.

Original: Taking the rope for example, the original environment is of size 128 × 128 × 128 .
Revised: Taking the rope for example, the original environment is of size 128 × 128 × 128 .

Original: However, a grid of size 32 × 6 × 32 is sufficient to cover the region of interaction between the environment and the rope.
Revised: However, a grid of size 32 × 6 × 32 is sufficient to cover the region of interaction between the environment and the rope.

Original: Therefore, we first detect the active region of size 32 × 6 × 32 , then we perform a linear transformation to shift the active region to the origin , perform the update due to deformation and interaction, finally we shift the updated active region to align with its original coordinates.
Revised: Therefore, we first detect the active region of size 32 × 6 × 32 , then we perform a linear transformation to shift the active region to the origin , perform the update due to deformation and interaction, finally we shift the updated active region to align with its original coordinates.

Original: Checkpointing method DaXBench only stores the values of a few states (i.e. checkpoints), and re-evaluates the sub-step gradients online during the back-propagation to reduce the memory consumption for long horizon problems.
Revised: Checkpointing method DaXBench only stores the values of a few states (i.e. checkpoints), and re-evaluates the sub-step gradients online during the back-propagation to reduce the memory consumption for long horizon problems.

Original: The checkpointing method has been proven useful in saving the memory consumption for differentiable simulations in many prior works (Qiao et al., 2021;
Revised: The checkpointing method has been proven useful in saving the memory consumption for differentiable simulations in many prior works (Qiao et al., 2021;

Original: Chen et al., 2016; Griewank & Walther, 2000).
Revised: Chen et al., 2016; Griewank & Walther, 2000).

Original: To make the simulator end-to-end differentiable, every value in the neural network that is computed during the forward pass has to be saved, as they are needed to compute the gradient during the backward pass.
Revised: To make the simulator end-to-end differentiable, every value in the neural network that is computed during the forward pass has to be saved, as they are needed to compute the gradient during the backward pass.

Original: The computation complexity scales up quickly w.r.t.
Revised: The computation complexity scales up quickly w.r.t.

Original: the length of the task horizon.
Revised: the length of the task horizon.

Original: This is a significant problem for our simulator for high
Revised: This is a significant problem for our simulator for high

Original: DoFs deformable object manipulation, as the memory required for each timestep’s update is enormous compared to its rigid body counterpart.
Revised: DoFs deformable object manipulation, as the memory required for each timestep’s update is enormous compared to its rigid body counterpart.

Original: To overcome the large memory consumption issue,
Revised: To overcome the large memory consumption issue,

Original: DaXBench trade-offs the computation complexity slightly for large memory consumption improvement: DaXBench only stores the values of a few states (i.e. checkpoints) at the forward pass; during the backward propagation, DaXBench re-compute segments of the forward values starting from every stored state in the reverse order sequentially for the overall gradient.
Revised: DaXBench trade-offs the computation complexity slightly for large memory consumption improvement: DaXBench only stores the values of a few states (i.e. checkpoints) at the forward pass; during the backward propagation, DaXBench re-compute segments of the forward values starting from every stored state in the reverse order sequentially for the overall gradient.

Original: With this trick, DaXBench
Revised: With this trick, DaXBench

Original: [Table]
Revised: [Table]

Original: now doubles the computation complexity, but saves an arbitrary amount of memory, depending on the intervals of the states saved.
Revised: now doubles the computation complexity, but saves an arbitrary amount of memory, depending on the intervals of the states saved.

Original: A. D ISCONTINUOUS G RADIENT
Revised: A. D ISCONTINUOUS G RADIENT

Original: As an end-to-end differentiable simulator
Revised: As an end-to-end differentiable simulator

Original: [Table]
Revised: [Table]

Original: the effectiveness of DaXBench rests on the accuracy of the gradient computed.
Revised: the effectiveness of DaXBench rests on the accuracy of the gradient computed.

Original: We found that for deformable object manipulation, the action not in contact with the object will cause the gradient to be discontinuous, which causes the estimate of the gradient after this action to suffer from high variance and poor accuracy in the backward propagation.
Revised: We found that for deformable object manipulation, the action not in contact with the object will cause the gradient to be discontinuous, which causes the estimate of the gradient after this action to suffer from high variance and poor accuracy in the backward propagation.

Original: To mitigate this problem, for every action input by the agent or the human user, DaXBench will perform local adjustment such that the action will always make the gripper in contact with the deformable objects.
Revised: To mitigate this problem, for every action input by the agent or the human user, DaXBench will perform local adjustment such that the action will always make the gripper in contact with the deformable objects.

Original: This ensures that our gradient is end-to-end continuous therefore improving the accuracy of the estimated gradient.
Revised: This ensures that our gradient is end-to-end continuous therefore improving the accuracy of the estimated gradient.

Original: B S IM -2-R EAL G AP
Revised: B S IM -2-R EAL G AP

Original: To verify that the dynamics of our simulated tasks have high fidelity and a small Sim2Real gap, we carry out a real robot experiment.
Revised: To verify that the dynamics of our simulated tasks have high fidelity and a small Sim2Real gap, we carry out a real robot experiment.

Original: We deploy CEM-MPC to the PushRope task on a Kinova Gen robot.
Revised: We deploy CEM-MPC to the PushRope task on a Kinova Gen robot.

Original: CEM-MPC uses our DaX simulator as the predictive model.
Revised: CEM-MPC uses our DaX simulator as the predictive model.

Original: Given a state, CEM-MPC plans the next best push action ( x, y, z, x ′ , y ′ , z ′ ) .
Revised: Given a state, CEM-MPC plans the next best push action ( x, y, z, x ′ , y ′ , z ′ ) .

Original: The state is estimated from a point cloud image, and the Cartesian space push action is transformed to the joint space trajectory via an inverse kinematics module.
Revised: The state is estimated from a point cloud image, and the Cartesian space push action is transformed to the joint space trajectory via an inverse kinematics module.

Original: Note we are not verifying the sim2real gap caused by the inaccuracy of state estimation or kinematics; we are interested in whether the dynamics of DaX can correctly guide the CEM-MPC to succeed in the task.
Revised: Note we are not verifying the sim2real gap caused by the inaccuracy of state estimation or kinematics; we are interested in whether the dynamics of DaX can correctly guide the CEM-MPC to succeed in the task.

Original: The experiment is included in our supplementary video.
Revised: The experiment is included in our supplementary video.

Original: C RL A LGORITHMS N UMERICAL E XPERIMENT R ESULTS
Revised: C RL A LGORITHMS N UMERICAL E XPERIMENT R ESULTS

Original: We report the numerical experiment performance for the RL algorithms in Table 4.
Revised: We report the numerical experiment performance for the RL algorithms in Table 4.

Original: This is to supplement the learning curve in the main text.
Revised: This is to supplement the learning curve in the main text.

Original: These numerical results report the final performance of the learning curves in Figure
Revised: These numerical results report the final performance of the learning curves in Figure

Original: [Table]
Revised: [Table]

Original: D T RANSPORTER WITH I MAGE AS I NPUT
Revised: D T RANSPORTER WITH I MAGE AS I NPUT

Original: We reported the performance of two different versions of Transporter in Table 5:
Revised: We reported the performance of two different versions of Transporter in Table 5:

Original: Transporter, which uses 3D particles and projects them onto an image to form a height map; and Transporter-RGB, which adds RGB channels and projects the particles onto the image plane, giving it white color with a black background.
Revised: Transporter, which uses 3D particles and projects them onto an image to form a height map; and Transporter-RGB, which adds RGB channels and projects the particles onto the image plane, giving it white color with a black background.

Original: We have found that an explicit process of rendering particles into RGB images significantly enhances the performance of Transporter.
Revised: We have found that an explicit process of rendering particles into RGB images significantly enhances the performance of Transporter.