UBSoft: A Simulation Platform for Robotic Skill Learning in Unbounded Soft Environments
Authors: Chunru Lin
Jugang Fan
Yian Wang
Zeyuan Yang
Zhehuan Chen
Lixing Fang
Tsun-Hsuan Wang
Zhou Xian
Chuang Gan
Venue: CoRL 2024 (2024)
Keywords: Soft-Body Manipulation
Locomotion
Physics Simulation

Abstract: It is desired to equip robots with the capability of interacting with various soft materials as they are ubiquitous in the real world. While physics simulations are one of the predominant methods for data collection and robot training, simulating soft materials presents considerable challenges. Specifically, it is significantly more costly than simulating rigid objects in terms of simulation speed and storage requirements. These limitations typically restrict the scope of studies on soft materials to small and bounded areas, thereby hindering the learning of skills in broader spaces. To address this issue, we introduce UBSoft, a new simulation platform designed to support unbounded soft environments for robot skill acquisition. Our platform utilizes spatially adaptive resolution scales, where simulation resolution dynamically adjusts based on proximity to active robotic agents. Our framework markedly reduces the demand for extensive storage space and computation costs required for large-scale scenarios involving soft materials. We also establish a set of benchmark tasks in our platform, including both locomotion and manipulation tasks, and conduct experiments to evaluate the efficacy of various reinforcement learning algorithms and trajectory optimization techniques, both gradient-based and sampling-based. Preliminary results indicate that sampling-based trajectory optimization generally achieves better results for obtaining one trajectory to solve the task. Additionally, we conduct experiments in real-world environments to demonstrate that advancements made in our UBSoft simulator could translate to improved robot interactions with large-scale soft material. More videos can be found at https://ubsoft24.github.io.

UBSOFT: A Simulation Platform for Robotic Skill Learning in Unbounded Soft Environments
Chunru Lin""; : Jugang Fan"", Yian Wang', Zeyuan Yang', Zhehuan Chen', Lixing Fang'. Tsun-Hsuan Wang", Xian Zhou', Chuang Gan 1.4
University of Massachusetts Amherst CCarnegie Mellon University
Maassacussetts Institute of Technology *MIT-IBM Watson Al Lab
Abstract: is desired to equip robots with the capability of interacting with various soft materials as they are ubiquitous in the real world. While physics simulations are one of the predominant methods for data collection and robot training, simulat- ing soft materials presents considerable challenges. Specifically, it is significantly more costly than simulating rigid objects in terms of simulation speed and storage requirements. These limitations typically restrict the scope of studies on soft mate- rials to small and bounded areas, thereby hindering the learning of skills in broader spaces. To address this issue, we introduce UBSOFT, a new simulation platform designed to support unbounded soft environments for robot skill acquisition. Our platform utilizes spatially adaptive resolution scales, where simulation resolution dynamically adjusts based on proximity to active robotic agents. Our framework markedly reduces the demand for extensive storage space and computation costs required for large-scale scenarios involving soft materials. We also establish a set of benchmark tasks in our platform, including both locomotion and manipulation tasks, and conduct experiments to evaluate the efficacy of various reinforcement learning algorithms and trajectory optimization techniques, both gradient-base and sampling- based.. Preliminary results indicate that sampling-basee trajectory optimization generally achieves better results for obtaining trajectory to solve the task. Additionally, conduct experiments in real-world environments demonstrate that udvancements made in our UBSOFT simulator could translate to improved robot interactions with large-scale soft material. More videos can be found at https: /vviivviii/// wwi.wwww.....................w.. cs. umass. edu/ wwweedduuuoteeeeee wwsoft/....////////////.o/......
Keywords: Soft-Body Manipulation, Locomotion, Physics Simulation
Introduction
Soft materials, such as sand and snow, are prevalent in daily life. Although humans routinely interact with these materials on large scale from building snowman to creating art sand beach-equipping robots with similar capabilities presents significant challenges. Unlike rigid objects and terrains that have been extensively studied in robotics and Embodied AI research, soft materials like sand and snow possess unique complexities. Their deformable nature leads to intractably high degrees of freedom, making them difficult to perceive and represent accurately. Moreover, their behavior may change depending on the scale at which they are observed, adding an additional layer of complexity. Moreover, its extremely high dimensionality also poses a challenge the complexity of computational methods including simulation and learning interact with such materials.
One prevalent method for teaching robots to manipulate and interact with these materials involves collecting extensive data to train the robots in virtual environments with physics solvers dedicated to simulating their physical behaviors [1, 2, 3, 4]. These models are then adapted for real-world applications. However, prior research was often restricted to narrowly defined tasks with confined
denotes equal contribution
8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

LBSOFT A SIMILTTION FLATFCRM PS ICOOTIC SKILL LEFRNING EVECUNDED SCFT ENVIRON ENTS
Figure 1: The title of the paper is written on a large scene covered with soft material by robotic manipulator.
workspace, limiting the task scope to small or simplistic manipulation challenges. For example, Huang et al. [3] focus on small tasks like block lifting or dough reshaping, Xian al. [4] address fluid manipulation tasks confined in container, Wang et al. [2] manipulate small pieces of cloth the table. Yet, these scenarios rarely scale to the complexities encountered in real-world environments; this requires either conducting multiple smaller-scale simulations in ad hoc manner and meticulously combining them together or, a more intelligent solution: large-scale yet efficient simulation. For instance, accurately simulating humanoid robot walking on a sand dune requires simultaneously modeling billions of sand particles, posing significant challenges in terms of both storage capacity and computational speed.
To address these challenges, several recent studies have attempted to simulate large-scale soft materials. A group of work aims to simulate extensive sand environments by using height field [5, 6]. However, this simplification can yield unrealistic results, especially for softer materials like snow which cannot be accurately modeled using 2D height field, or when interaction extends beneath the terrain surface (e.g., deep excavation) involved. Other research efforts [7] try to mitigate the impact on computational resources by simply freezing particles that are far from agents. Nevertheless, this approach overlooks the fact that material distant from the agent still gradually affect the behaviors of both the agent and the environment. Additionally, this strategy does not fully resolve the issues of storage capacity required for large-scale simulations. Yue et al. [8] developed more efficient method by adaptively coupling discrete and continuum simulations of granular materials. Despite these advancements, simulating even just within the continuum space remains a costly endeavor. As a result, there still remains substantial gap in the current literature concerming the simulation of large scenarios filled with soft materials.
still move, albeit slowly, and
To this end, present UBSOFT, simulation platform aiming to efficiently simulate large-scale scenes filled with soft materials, enabling real-time and morihorizon robot learning in unbounded environments. UBSOFT is fully differentiable physics simulation platform for robotic skill learning, and supports simulation of giant scenes such as an agent writing the paper title on the sand in Fig. I (left) and robot dog walking on desert Fig. (right). We use Material Point Method (MPM) [9], a hybrid particle-grid method to simulate soft materials, and propose a spatially adaptive simulation scheme where simulation resolution dynamically adjusts based on their proximity to the active robotic agent. This enables both efficiency in simulation speed and memory cost, while still leading to accurate simulation behavior of the whole environment. Specifically, we implement hierarchical grid system and a corresponding particle adjustment mechanism: finer grids and smaller particles are utilized in the region closer to the robot to increase detail and accuracy, while larger grids and particles are used further away to conserve computational resources. addition, we also present a suite of long-horizen tasks covering both locomotion tasks and manipulation tasks that demonstrate the capability of our platform. Our simulation platform implemented to be fully differentiablee and grants access to gradient information of the system, which has been proven useful for optimizing robot policies when interacting with soft materials in various literature [3, 4, 10]. We conducted evaluations using gradient-basse and sampling-based trajectory optimization methods, as well as state-of-the-an reinforcement learning algorithms. The findings indicate that sampling-bas based trajectory optimization generally delivers the best performance in our context. Reinforcement learning algorithms, which map states to actions, struggle with partial and high-dimensional state representations. Meanwhile, gradient-based methods are less effective on their own due the highly non-convex landscape and inaccuracies in long-term mack-rropagation We summarize contributions follows:

---

We introduce UBSOFT, a simulation platform designed to facilitate robot learning in spatially exten- sive environments populated with diverse soft materials. This platform addresses the critical need for comprehensive training in both manipulation and locomotion tasks in expansive environments.
The core of UBSOFT is spatially adaptive physies engine that efficiently models various soft materials in large and potentially unbounded environments. We enhance this capability by integrating interactions with rigid materials and incorporating the differeentiaility of our simulation engine, further supporting the training of robotic agents.
We present suite of tasks to benchmark reinforcement learning algorithms and optimization- methods to comprehensively analyze the performance and challenges of current methods.
2
Related Works
Physics Simulation for Soft Environment. Recent scholarly interest increasingly focuses on the integration of differentiable simulation and machine learning due its ability to provide specific environmental feedhack through physics. A prevalent approach for building differentiable simulations involves learning forward dynamic models via neural networks [11, 12, 13, 14, 15, 16, 17]. which allows for direct gradient propagation through the networks. Although this method typically simulates faster than real-time, it faces challenges such as data dependency, sim-to-real gap, and limitations in handling
scenarios [18]. Another widely adopted method involves implementing physical simulations differentiable manner, as seen [19, 20, 3, 4, 21, 22, 23, 24, 25J. Here, differentiabilityy is achieved using automatic differentiation tools like Taichi [19] and Nvidia Warp [26]. In studies such as Du et al. [27], Li et al. [28], Wang et al. [2], gradients are derived by explicitly computing analytic gradients and back propagating through the simulation process. These gradient data have been shown to be valuable for tasks that involve the control and manipulation of both rigid
and soft bodies [21, 29, 30, 31, 32, 33].
Interaction with Soft Material. Robotic manipulation with soft materials has been greatly investi-
gated recently, including manipulation with co-dimensional materials [34, 35, 36, 37, 38, 2, 39, 40], elastoplastic materials [1, 3, 33, 30, 24, 41, 42, 43], granular materials [44, 45], and fluid [4, 46, 47, 48]. Due to the lack of ability to simulate large scenarios, all those works can only deal with tasks in small scope. There are also groups of works interacting with soft terrains, Wang et al. [32] simulate soft terrain, however, it's still confined within a small scope. Alvarado et al. [49] model soft terrain with only stress-strain model to generate realistic footprints but doesn't 't deliver real physics. Zhu et al. [5] simulate sand with a height field to achieve sand-vehicle interactions, but it's only for shallow sands and doesn't solve the scaling problem. As a result, we aim build a simulation platform that can effectively simulate large scenarios in all three dimensions.
3 Spatially Adaptive Physics Engine
We develop our physics engine using Taichi [19, 50), a domain-specific programming language embedded in Python for engine supports various materials, including solid materials (rigid, plastic), soft materials (sand, snow, elastoplastic), and the interactions among them. To enable large-scale and even unbounded soft materials simulation, we design and implement novel spatially adaptive scheme on top of the Material Point Method, which draws inspiration from the octree structure and is capable of adjusting the simulation precision for different spatial regions based on the robot agent's 's location.
parallel programming with efficient auto-diff tools. The
Differentiability and Rendering Following Huang et al. [3], our simulation engine is implemented based on Taichi's s autodiff system compute gradients for simple operations and also implemented an analytical gradient for svd operation [51]. The gradient checkpointing we implemented supports gradient flow over long temporal horizons. UBSOFT also comes with a real-time OpenGL-based renderer and a path tracing renderer implemented using Luisa compute [52].
3

---

Robot Agent Previous Position
Robot Agent Current Position
Hierarchical Particles
Hierarchical Grids
Split
Rcsamole
P2G
G2P
Particles
Merge
Particic Positions
Particlo Positions
Grid Momentums
New Particle Positions
MPM Simulation with Spatially Adaptive Scheme. Hierarchical grids centered around the robot agent and become sparser further out. As the robot moves, these grids move in syne, and the particles are resampled match the grid cell sizes by splitting large particles in small grids and merging small particles in large grids. Subsequent phases of P2G and G2P adhere to standard MPM protocols but are adapted accommodate hicrarchical grids and particles.
Figure 2:
3.1 Material Modeling
Soft body. We represent soft materials with both Lagrangian particles and Eulerian background grids, using the Moving Least Squares Material Point Method (MLS-MPM) [9]. Specifically, the material properties such as position, velocity, density, volume, and deformation gradients are stored in the Lagrangian particles while the contact with the rigid body and the particle interactions are handled in Eulerian background grids. While our simulation could handle any soft materials that can be simulated by MLS- MLS-MPM, we highlight some examples used in our experiments. For instance, sand simulated using a Drucker-Prager plasticity model, which accounts for the material's yield stress and flow rules, effectively capturing the granular flow behavior [53]. Similarly, snow is modeled using combination of elasticity and plasticity represent its unique compaction and melting characteristics under different conditions [54].
Rigid body. In our simulation engine, we employ two types of geometric representations to facilitate the inclusion of rigid bodies, each capable of bidirectional coupling with soft variable structures. The first type, termed "Manipulator". uses Signed Distance Fields (SDFs) model individual rigid body parts like end-effectors. This method supports differentiable simulation, making it suitable for manipulation tasks. The second type, known as "Articulator". is designed to handle complex articulated rigid bodies featuring multiple links and joints, such as Quadruped robots.
3.2 Spatially Adaptive Scheme
propose a spatial adaptation scheme designed enhance the efficiency our simulations. Our rationale is based on the observation that only the immediate vicinity of the robot agent requires high-fidelity simulation, this constitutes small portion of the overall expansive environment. For the vast majority of the environment, which lies distant from the robot, general overview of movement trends along with rough records resulted from past interaction suffices, eliminating the need for granular detail. Consequently, our strategy focuses primarily on the local environmental information around the robot agent while retaining global environmental information about the footprint from past robot's interaction. Centered on the robot agent, our approach involves employing higher-resolution simulations for proximate areas and loworriesolution simulations for regions that further away and changing the resolution accordingly as the movement of the robot agent.
Hierarchical grids. The granularity of Material Point Method (MPM) simulations depends on the fineness of spatial discretization; finer granularity, achieved through smaller grid cell sizes, results in higher simulation accuracy. Therefore, we introduce a hicrarchical grid system as illustrated in Fig.2. Centering on the robot's position, grids of varying granularity are nested, enabling spatial discretization varying degrees based on the robot's position. Unlike traditional MPM, which applies uniform granularity across the entire spatial domain, hierarchical grid adapts dynamically according to the distance to the agent. Specifically, with the smallest grid's d's side length lo we'l 'Il define

---

Sand Painting
Scoop Out
Dig A Hole
Smooth Surface
a Manipulation
Quadruped Sand Walk
Quadruped Snow Walk
Humanoid Stand
Humanoid Walk
(b) Locomotion
Figure 3: 4 manipulation tasks and 4 locomotion tasks proposed in UBSOFT.
series of grids with the side length 2o Also, set the number of the grid each dimension to 2 * h; in this way, assume the agent centering at (ar, 2). the smallest grids will fill in the area from (x klo.
klo, 2 klo) to (rr + klu, + klo, + klo) and subsequently the grids with scale 4 would fill in (x - kl;, kd,, kl.) to (ær + kls, + kl, + k4);; meanwhile the inner square still occupied by smaller grids.
Hierarchical particles. To maintain average number of partieles per grid cell and also to further save storage space when storing particles, also adapt the particle size following the grid size. This requires particle split and particle merge process. Particle split: Due to the simultaneous movement of particles and the grid, when large particles appear within small grid cells, they will be split into several smaller particles matching the grid cell size to ensure simulation accuracy. The positions these smaller particles are resampled based on the position the large particle, while conserving the momentum before and after the splitting, as shown in Fig. 2 Split. Particle merge: When certain number of small particles are present within large grid cell, they are merged into larger particle to save computational resources. The position of this larger particle is determined by the centroid of the small particles and other maintained information on the particles is averaged according to specific rules, as shown in Fig. 2 Merge. Readers are recommended to refer to the code and Appendix for detailed implementation.
4 Benchmark Tasks
4.1 Manipulation Tasks
UBSOFT contains 4 diverse manipulation tasks as presented in Fig. 3 where an agent interacts realistically with specific soft material using a tool. Details each task are described as follows.
Sand Painting. This task requires the final surface of the sand to display a heart shape. Therefore, the agent needs move the tool stirrer, to stir the sand and achieve the specified goal pattern.
Scoop Out. duck partially buried in the sand needs to be scooped out by the agent using shovel. This task is particularly challenging since the agent must carefully plan the movement path. Otherwise, it is easy to push the duck deeper into the sand.
Dig A Hole. The agent needs to mimic human-like manipulation of the shovel complete parabolic-like like trajectory, thereby digging hole in the flat sand surface.
5

---

NLS-MPM Ours
Particie Count by Scene Scale
FPS by Scene Scale
Number Particies
Frammen per Secord
-23
08
1.5
2.3
3.
1.0
1.5
3.0
30
Figure 4: With larger scenes, the spatially adaptive scheme significantly reduces the storage space
required and maintains a higher simulation speed. The dotted lines are extrapolated results where transitional MPM fails to simulate.
Simulation
# particles
time(s)
FPS f
Chamfer distance
MLS-MPM 256
11,612,880
579.9
8.6
MLS-MPM 128
1,444,000
37.1
134.8
12.97
Ours
712,250
27.4
182.5
5.32
MLSSMPM 64
156,620
8.2
609.7
14.15
MLS-MPM 32
16,810
6.6
769.2
15.58
Table 1:
Comparing our spatially adaptive simulation with MLS-MPM of different resolutions.
Smooth Surface. Given sand heap with an overall flat surface but distinct protrusion, the agent must accurately identify the raised area and move the scraper approach it. Then, through gentle pressing other smoothing actions, the agent finally smooth the surface of the sand.
4.2 Locomotion Tasks
Humanoid Stand. This task requires a humanoid robot to maintain standing on a sand landscape. The robot must continuously adjust its balance to prevent falling, assessing its ability to adapt to the sandy terrain and maintain stability over an extended period.
Quadruped (Humanoid) Walk. A quadruped (humanoid) robot is required to continuously walk on an unbounded sandy pr snowy landscape, resembling desert. The primary goal is to ensure the robot keeps moving forward while maintaining balance and avoiding getting stuck. To achieve this, the robot must constantly adjust its gait and posture to adapt to the sand's or snow's 's resistance.
5 Experiments
We conduct extensive experiments to answer the following questions:
How is our proposed simulator compared with existing simulators for soft materials?
How do different learning algorithms and optimization-basel methods perform on our
UBSoft benchmark?
Can policy acquired from the UBSoft Simulator transfer the real world?
5.1 Comparisons with State-of-the-art Simulators
Existing simulators for soft materials mostly rely on MIS-MPM or MPM for simulation. MLSS- MPM represents an advancement over MPM by introducing a novel stress divergence discretization, allowing it to twice fast as MPM. For detailed comparison of the speed benchmarking between MPM and MLS-MPM paper[9]. Given its faster and more stable performance, we have chosen MLS-MPMM as our baseline simulation method.
recommend readers refer to Section 6.1 of the MLS-MPM
To assess the efficiency of our simulator, we conducted the first experiment to simulate scenes of different scales with MLS-MPM and our spatially adaptive simulation respectively. We monitored the number of particles throughout the simulation process to track the increase in maximum storage required as the scene's spatial scale expanded. Additionally, benchmarked the speed of our

---

PPO
SAC
CHNEES
Draw
Scop Out
Dig
Smooth Surface
2310
1200
-20000
Loces
-309000
-0ces
3000
20
123
8:3
303
1009
126
623
1003
202
201
603
1010
233
450
us
100
Quadruped Walk Sand
Quadrupad Walk
Humanoid Stand
Humanoid Walk
3000
1500
52k3
2000
sods
Figure 5: Reward curves for all methods including PPO, SAC, CMA-ES, and Differentiable Physics.
Task
Sand Painting
Scoop Out
Dig A Hole
Smooth Surface
Hcuristic
783.4
208.1
2038.8
PPO
-17280.3 415.4
1466.7 + 22.6
980.1 # 11.2
7865.2 + 725.6
SAC
-3048.2 93.9
1265.9 161.0
1595.6 = 48.7
7610.4 + 998.7
CMA-ES
-2860.3 45.7
1573.2 + 5.7
1067.0 99.0
6844.8 40.7
DiffPhys
4000.0 # 37.8
1251.0 + 65.7
393.1 # 0.4
3269.0 59.6
Task
Quadruped Sand Walk
Quadruped Snow Walk
Humanoid Stand
Humanoid Walk
PPO
862.8++94.8
2387.44+336.7
528.3+332.1
518.8+32.7
SAC
1572.811.45.2
1225.01:201.8
553.1134.7
475.
CMA-ES
1037.9+123.3
1160.6::95.9
717.7+44.77
524.5=19.3
Table 2: The final accumulated reward and the standard deviation for each method. simulator against MLS-MPM, As in Fig. 4, method significantly reduces the storage space required for larger scales and achieves a tenfold speedup over MI.S-MPH as the scene scale increases.
To further evaluate the realism of our simulator, we conducted a comparative analysis between our spatially adaptive simulator and MLS-MPM of different resolutions (256, 128, 64, 32). The simulated scene is rigid cube of size 0.1 x 0.1 0.1 falling into a 1 1x 1 0.2 sand and the simulation runs 5000 timesteps. Although simulation realism is challenging to quantify precisely, in the field of simulation, finer granularity yields higher accuracy, bringing results closer reality. Therefore, we used the MLS-MPM 256 simulation as the reference and calculated the Chamfer distance between the particle sets to quantitatively assess the realism of our simulation. The results are presented in Table I ranked by time and space consumption. UBSOFT achieved a reduction in time and space consumption by over an order of magnitude. Furthermore, our simulation closely matches the performance of MLS-MPM 256, while requiring only half the space and less time than MLS-MPM 128.
5.2 Benchmark different policy learning methods
Our goal is to establish foundation for skill learning in large-scale environments filled with soft materials. Leveraging our simulation platform, we introduce several tasks tailored to this context and benchmark existing algorithms to broadly assess their effectiveness in these tasks. These experiments will lay the groundwork for future research on solving tasks involving large-scale soft materials.
We benchmark several methods for manipulation and locomotion tasks proposed in section These include gradient- madient--baed trajectory optimization method using differentiable physics (DiffPhys), sampling based trajectory optimization method known as Covariance Matrix Adaptation Evolution Strategy (CMA-ES) [55]. model-free Reinforcement Learning (RL) algorithms such as Soft Actor- Critic (SAC) [56] and Proximal Policy Optimization (PPO) [57], and a heuristic agent manually designed by the authors. For the implementation of these algorithms, we utilize the ttable-baselines library [58] for RL. algorithms and refer to [59] for the implementation of CMA-ES. Notably, for
locomotion tasks, we do not use DiffPhys the experiments due its current limitation in handling gradients for articulated objects. Following Wang al. [2], compare the reward of the best trajectory of different methods.
7

---

Implementation Details. For manipulation tasks, we use a 4-layer grid of size 4 x 64 with resolutions of 256, 128, 64,32 the spatial scale x x 0.2, and the simulation step is set to 0.002s. We use an 8-layer grid of size 8 x 32 with resolutions from 128 to and a spatial scale of 10x 10x0.5 for the locomotion tasks.
Evaluation metrics. We utilize human-defined reward function as the direct evaluation metric because it most accurately reflects the training objectives. Rewards are designed based on Chamfer distance for Sand Painting, height variation and scooped particles for Dig A Hole and Scoop Out, and particles above a threshold for Smooth Surface. While for various locomotion challenges, the reward is derived directly from the state the rigid bodies. The detailed reward function design is elaborated in Appendix B.3.
Discussions. Generally, the heuristic agents struggle to finish the task compared to other agents, even with extensive human involvement. RL algorithms generally struggle with manipulation tasks. This difficulty arises primarily because soft materials have infinite degrees of freedom, which are downsampled to create the observation space. Thus, the observation space cannot fully capture the current state, especially when it needs to represent subtle geometries such as Sand Painting.
Compared reinforcement learning algorithms, the sampling- based trajectory optimization algorithm performs generally better. This is because reinforcement learning algorithms learn policy that maps states to actions, where the state space is very partial and of high dimensionality. In contrast, CMA-ES only learns a trajectory that achieves the best performance starting from the initial state, which can be much easier than learning an entire policy.
We find that directly applying gradient-based optimization using differentiable physics simulation is less effective. The gradient becomes inaccurate due to clipping during long-term pock-propagation. However, when it comes the task Sand Painting that requires subtle manipulation skills to achieve and have continuous contacts, the differentiable physics-based trajectory optimization method would outperform others.
5.3 Sim-to-Real Transfer
We selected the tasks of sand painting and scooping establish real-world experiments using an XArm, exploring open-loop transfer. For each task, we first build the digital twins of the real-world scenario in our simulator, by meticulously adjusting the sand model with parameter of Young's modulus E = 1e6, Poisson's ratio v = 0.2, density p = 1000.0 and friction angle a = 45 to align the observation in the simulation and in the real-world. After that, we optimize the trajectory of the manipulator our simulated environments and then directly deploy on the real robotic arm to replicate the trajectory we have in the simulation. We find our trajectory optimized in the UBSoft simulator works well in the real world. As shown in Appendix, the robotic arm successfully writes the letters "CoRL." in the sand, and scoops out the cube from the sand, proving the successful open-loop transfer from UBSoft simulator, which indicates the realistic and potential of our simulator with small sim-to-real gap. Furthermore, advancements achieved in UBSOFT may also translate into tangible improvements for robots operating in environments dominated by expansive soft materials.
6 Conclusion
We introduced UBSOFT, a new simulation platform for benchmarking robot manipulation and locomotion skills learning in large environments filled with diverse soft materials. At the heart of UBSoFT is a spatially adaptive physics engine that efficiently simulates soft materials in extensive, potentially unbounded environments. This engine utilizes an octree structure that dynamically adjusts to the robot's movements. We have further enhanced the platform by enabling interactions with rigid materials and incorporating the differentiability of our simulation to aid in robotic skill learning. The designed manipulation and locomotion tasks rigorously evaluate the performance of existing methods, providing insights for the development of more capable and resilient robots in expansive settings.
Acknowledgment This project was supported by NSF IIS- IIS-2404386.

---

References
[1] Y. Wang, Z. Xian, F. Chen, T.-H. Wang, Y. Wang, K. Fragkiadaki, Z. Erickson, D. Held, and
C. Gan. Robogen: Towards unleashing infinite data for automated robot learning via generative simulation. arXiv preprint arXiv:2311.0145. 2023,
[2] Y. Wang. J. Zheng, Z. Chen, Z. Xian, G. Zhang, C. Liu, and C. Gan. Thin-shell object manipulations with differentiable physics simulations. The Twelfth International Conference on Learning Representations, 2023.
[3] Z. Huang, Y. Hu, T. Du, S. Zhou, H. Su, J. B. Tenenbaum, and C. Gan. Plasticinelab: A soft- body manipulation benchmark with differentiable physics. arXiv preprint arXiv:2104.03311, 2021.
[4] Z. Xian, B. Zhu, Z. Xu, H.-Y. Tung, A. Torralba, K. Fragkiadaki, and C. Gan. Fluidlab: differentiable environment for benchmarking complex fluid manipulation. arXiv preprint arXiv:2303.022:: 02344, 2023.
[5] K. Zhu, X. He. S. Li, H. Wang, and G. Wang. Shallow sand equations: Real-time height field simulation of dry granular flows. IEEE Transactions on Visualization and Computer Graphics, 27(3):2073-2884, 2019.
[6] B. Zhu and X. Yang. Animating sand as surface flow. In Eurographics (short papers), pages 9-12, 2010.
[7] M. Ihmsen, A. Wahl, and M. Teschner. High-resolution simulation of granular material with sph. In VRIPHYS, pages 53-60, 2012.
Y. Yue, B. Smith, Y. Chen, M. Chantharayukhonthonn, K. Kamrin, and E. Grinspun. Hybrid grains: Adaptive coupling of discrete and continuum simulations of granular media. ACM Trans. Graph., 37(6))283:1 1-12288:1122881 183:19,, Nov. 2018.
[9] Y. Hu, Y. Fang, Z. Ge, Z. Qu, Y. Zhu, A. Pradhana, and C. Jiang. A moving least squares material point method with displacement discontinuity and two-way rigid body coupling. ACM Transactions on Graphics (TOG), 3714) 2018.
[10] Z. Xu, Xian, X. Lin, C. Chi, Z. Huang, Gan, and S. Song. Roboninja: Learning an adaptive cutting policy for multi-material objects. arXiv preprint arXiv:2302.1 1553, 2023.
[11] J. Han, W. Huang, H. Ma. J. Li, J. Tenenbaum, and C. Gan. Learning phys- ical dynamics with subequivariant graph neural networks. In S. Koyejo, S. Mo- hamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neu- ral Information Processing Systems, volume 35, pages 26256 26268. Curran Associates, Inc., 2022. URL http:sthttt//ppooeeeeeeeeee 88445f6c366775557002188888888888
www.wew....aaapppppeee cctpccer_ppeeeeeeeeeeeeee files/ www.leeeppppaeee https//eleeeeeee/ee//////e /aper//0222/Eile Conf-ronneeeeeeeeeeeeeee pdf.
[12] Y. Li, J. Wu, R. Tedrake, J. B. Tenenbaum, and A. Torralba. Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids. arXiv preprint arXiv: 181000160,. 2018.
[13] T. Pfaff, M. Fortunato, A. Sanchez-Gonzalez, and P. W. Battaglia. Learning mesh-based simulation with graph networks. arXiv preprint arXiv:2010.03409, 2020.
[14] C. Schenck and Fox. Spnets: Differentiable fluid dynamics for deep neural networks. In Conference on Robot Learning, pages 317-335. PMLR, 2018.
[15] B. Ummenhofer, L. Prantl, N. Thuerey, and V. Koltun. Lagrangian fluid simulation with continuous convolutions. In International Conference Learning Representattonss 2019.

---

[16] Z. Xian, S. Lal, H.-Y. Tung, E. A. Platanios, and K. Fragkiadaki. Hyperdynamics: Meta-learning object and agent dynamics with hypernetworks. arXiv preprint arXiv:2103.09433 2021.
[17] J. Han, W. Huang, H. Ma, Li, J. B. Tenenbaum, and C. Gan. Learning physical dynamics with subequivariant graph neural networks. Thirty-Sixth Conference on Neural Information Processing Systems, 2022.
[18] A. Sanchez-Gonzalez J. Godwin, T. Pfaff, R. Ying, J. Leskovee, and P. Battaglia. Learning to simulate complex physics with graph networks. In International conference on machine learning, pages 8459-8468. PMLR, 2020.
[19] Y. Hu, L. Anderson, T.-M. Li, Q Sun, N. Carr, J. Ragan-Kelley, and F. Durand. Difftaichi: Differentiable programming for physical simulation. arXiv preprint urXiv: 2019.
[20] de Avila Belbute-Peres, K. Smith, K. Allen, J. Tenenbaum, and J. Z. Kolter. End-to-end differentiable physics for learning and control. Advances in neural information processing systems, 31, 2018.
[21] Y.-L. Qiao, J. Liang, V. Koltun, and M. C. Lin. Scalable differentiable physics for learning and control. arXiv preprint arXiv:2007.02 168, 2020.
[22] Y.-L. Qiao, J. Liang, V. Koltun, and M. C. Lin. Differentiable simulation of soft multi-body systems, In Conference on Neural Information Processing Systems (NeurIPS), 2021.
[23] S. Chen, Y. Xu, C. Yu, L. Li, X. Ma, Z. Xu, and D. Hsu. Daxbench: Benchmarking de- formable object manipulation with differentiable physics. In The Eleventh International Con- ference on Learning Representations, 2023. URL htR.hhtss:///peeeeeee http:///peeeeeeeeeeeeeee www.iiiwwwweetfeee nwt/:/rmmfffe 1NAzMofMmW1.
[24] Li, Z. Huang, Chen, T. Du, H. Su, B. Tenenbaum, and C. Gan. Dexdeform: Dexterous deformable object manipulation with human demonstrations and differentiable physics. arXiv preprint arXiv:
2023.
[25] X. Zhu, J. Ke, 7. Xu, 7. Sun, B. Bai, Lx, Q. Liu, Y. Zeng, Q Ye, C. Lu, et al. Diff-Ifd: Contact-aware model-based learning from visual demonstration for robotic manipulation via differentiable physics-basee based simulation and rendering. Conference on Robot Learning, pages 499-512. PMLR, 2023.
[26] M. Macklin. Warp: A perfrorrfore python framework for gpu simulation and graphics. In NVIDIA GPU Technology Conference (GTCC, 2022.
[27] T. Du, K. Wu, P. Ma, S. Wah, A. Spielberg, D. Rus, and W. Matusik. Diffpd: Differentiable projective dynamics. ACM Transactions on Graphics (TOG), 41(2):1-21, 2021.
[28] Y. Li, Du, K. Wu, J. Xu, and W. Matusik. Diffcloth: Differentiable cloth simulation with dry frictional contact. ACM Trans. Graph., 42(1), oct 2022. ISSN 0730-0301. doi:10.1145/352766 10.
URL https: //doi. 1ttg//..115//52766/
[29] J. Xu, T. Chen, L. Zlokapa, M. Foshey, W. Matusik, S. Sueda, and P. Agrawal. An end-to-end differentiable framework for contact-aware robot design. arXiv preprini urXiv:2107.07501, 2021.
[30] X. Lin, Z. Huang. Y. Li, J. B. Tenenbaum, D. Held, and C. Gan. Diffskill: Skill abstraction from differentiable physics for deformable object manipulations with tools. arXiv preprint arXiv:2203. 17275. 2022.
[31] S. Li. Z. Huang, T. Du, H. Su, J. B. Tenenbaum, and C. Gan. Contact points discovery for soft-body manipulations with differentiable physics. arXiv preprint arXiv:2205.02888 2022.
10

---

[32] T.-H. Wang, P. Ma, A. E. Spielberg, Z. Xian, H. Zhang, J. B. Tenenbaum, D. Rus, and C. Gan. Softzoo: A soft robot co-design benchmark for locomotion in diverse environments. arXiv preprint arXiv:2303.09555 2023.
[33] Z. Huang, F. Chen, Y. Pu, C. Lin, H. Su, and C. Gan. Diffvl: Scaling up soft body manipulation using vision-language driven differentiable physics. Advances in Neural Information Processing Systems, 36, 2024.
[34] C. Chi, B. Burchfiel, E. Cousineau, Feng, and S. Song. Iterative residual policy for goal- conditioned dynamic manipulation of deformable objects. In Proceedings of Roboties: Science and Systems (RSS), 2022.
[35] C. Zhao, C. Jiang, J. Cai, M. Y. Wang, H. Yu, and Q. Chen. Flipbot: Learning continuous paper flipping via coarse-to-fine ixterocepive-proorroeppp
exploration. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 10282-10288. IEEE, 2023.
[36] H. Ha and S. Song. Flingbot: The unreasonable effectiveness of dynamic manipulation for cloth unfolding. In Conference on Robotic Learning (CoRL), 2021.
[37] Z. Xu, C. Chi, B. Burchfiel, Cousineau, S. Feng, and S. Song. Dextairity: Deformable manipulation can be breeze. In Proceedings of Robotics: Science and Systems (RSS), 2022.
[38] R. Wu, C. Ning, and H. Dong. Learning foresightful dense visual affordance for deformable object manipulation. In IEEE International Conference on Computer Vision (ICCV), 2023.
[39] M. Sudry, T. Jurgenson, A. Tamar, and E. Karpas. Hierarchical planning for rope manipulation using knot theory and a learned inverse model. In Conference on Robot Leurning, pages 1596-1609, PMLR, 2023.
[40] N. Sunil, Wang. She, E. Adelson, and A. R. Garcia. Visuotactile affordances for cloth manipulation with local control. In Conference Robot Learning, pages 1596-1606. PMLR, 2023.
[41] H. Shi, H. Xu, S. Clarke, Y. Li, and J. Wu. Robocook: Long-horizon elasto- caastt-plattcc object manipulation with diverse tools. arXiv preprint arXiv:2306. 14447, 2023.
[42] X. Lin, C. Qi, Y. Zhang, Z. Huang, K. Fragkiadaki, Y. Li, C. Gan, and D. Held. Planning with spatial-temporal abstraction from point clouds for deformable object manipulation. arXiv preprint ar 2rXiv:2210.1577 2022.
[43] M. Oller, M. P. Lisbona, D. Berenson, and N. Fazeli. Manipulation via membranes: High- resolution and highly deformable tactile sensing and control. In Conference Robot Learning, pages 1859-1859. PMLR, MMLR,2 2023.
[44] D. Millard, D. Pastor, J. Bowkett, Backes, and G. S. Sukhatme. Granular gym: High perfor- mance simulation for robotic tasks with granular materials. arXiv preprint arXiv:2306.0/ 2023.
[45] A. P. Thompson, H. M. Aktulga, R. Berger, D. S. Bolintineanu, W. M. Brown, P. S. Crozier, P. J. In't Veld, A. Kohlmeyer, S. G. Moore, T. D. Nguyen, et al. Lammps-a flexible simulation tool for particle-based materials modeling at the atomic, meso, and continuum scales. Computer Physics Communications, 271:108171. 2022.
[46] M. Sieb, Z. Xian, A. Huang, O. Kroemer, and K. Fragkiadaki. Graph-structured visual imitation. In Conference on Robot Learning, pages 979-989, PMLR, 2020.
[47] D. Seita, Y. Wang, S. J. Shetty, E. Y. Li, Z. Erickson, and D. Held. Toolfiowne:: Robotic manipulation with tools via predicting tool flow from point clouds. In Conference on Robot Learning, pages 1048-0049. PMLR, 2023.

---

[48] Y. Li, S. Li, V. Sitzmann, P. Agrawal, and A. Torralba. 3d neural scene representations for visuomotor control. In Conference on Robot Learning, pages 112-123. PMLR, 2022.
[49] E. Alvarado, C. Paliard, D. Rohmer, and M.-P. Cani. Real-time locomotion soft grounds with dynamic footprints. Frontiers in Virtual Reality, 3:801856, 2022.
[50] Y. Hu, Fang, Ge, Z. Qu, Y. Zhu, A. Pradhana, and C. Jiang. moving least squares material point method with displacement discontinuity and two-way rigid body coupling. ACM Trans. Graph., 37(4), jul 2018. ISSN 0730-0301. doi:10.1 145/3197517.3201299
[51] J. Townsend. Differentiating the singular value decomposition. Technical report, Technical Report 2016, https://j-towns. github. io/papers/svd-derivatvs .... 2016.
[52] S. Zheng, Z. Zhou, X. Chen, D. Yan, C. Zhang, Y. Geng, Y. Gu, and K. Xu. Luisarender: A high- performance rendering framework with layered and unified interfaces on stream architectures. ACM Trams. Graph., (6), nov 2022. ISSN 0730-0301. doi: 05001444555444455555533 URL
https://doli http://t.ioo.rgo....//////
3555463.
[53] G. Klár, т. Gast, A. Pradhana, C. Fu, C. Schroeder, C. Jiang, and J. Teran. Drucker-prager elastoplasticity for sand animation. ACM Trans. Graph., 35(4), jul 2016. ISSN 0730-0301.
do::10..14452299924222255666 URL https::/tpoo. http:///doooooooo///o...oo / 1011/1129//22 29959966
[54] A. Stomakhin, C. Schroeder, L. Chai, J. Teran, and A. Selle. A material point method for snow simulation. ACM Transactions Graphics (TOG), 32(4): 2013.
[55] N. Hansen and Ostermeier. Completely derandomized self-adaptation evolution strategies. Evolutionary computation, 9(2:159-199, 2001.
[56] T. Haarnoja, A. Zhou, P. Abbeel, and Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pages 1861-1870. PMLR, 2018.
[57] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 1707.06347, 2017.
[58] A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann. Stable-baselines3: Reliable reinforcement learning implementations, The Journal of Machine Learning Research, 22(1):12348-122)
2021.
[59] N. Hansen, Y. Akimoto, and P. Baudis. Cma-es/pyema on github. zenodo, doi: (feb.. 2019), 2019.
12

---

A Implementation Details on the UBSorTEngine
Figure 6: Hierarchical Grids for Spatially Adaptive Scheme.
Figure 7: A robot dog walks in desert with dunes.
A.1 Spatial Adaptive Scheme
To improve the efficiency of our proposed algorithm, implement dynamic data structure based on tagging and amortized restructuring to support the deletion and addition of particles during the resampling process.
The hierarchical grid system introduced in section 3.2 illustrated in Figure 6. Centering on the robot's position, grids of varying granularity are nested, enabling spatial discretization varying degrees based on the robot's 's position. Unlike traditional MPM, which applies uniform granularity across the entire spatial domain, our hierarchical grid adapts dynamically according to the distance to the agent.
With the proposed spatial adaptive scheme, UBSOFT is able to efficiently simulate large-scale scenes filled with soft materials, enabling real-time and long-horizon robot learning in unbounded environments, such robot dog walking on a desert in Figure 7.
B UBSOFT Tasks and Evaluation Details
B.1 Task Representation
Task Formulation. We design
of tasks: manipulation tasks and locomotion tasks in our task collection. The two types of task share the same formulation, finite-horizon Markov Decision Process (MDP). which contains state space S, action space A, reward function R S x AA x S -R, and transition function T x A S. The transition function is determined by the forward simulation. The loss and reward function for each task is discussed Section 5. The goal of the robot agent is to find policy π (als) that produces action sequences to maximize the expected total return Ex EE 7 (R(s,, where E (0, 1) is the discount factor and T is the horizon of the task.
State Space and Action Space. In manipulation tasks, we suppose the robot agent is robot arm with an end-effector and the state space SA + Sp includes two parts. SA E 2g represents the pose and velocity information of the end-effector of the robotic arm in the scene, where d is the degree of freedom in the specific task. Sp N x6 is the deformation state of non-rigid objects with
13

---

particle-base representation, where Np is the max possible number of particles in the simulation using our Spatially Adaptive MPM method and 6 represents the 3D position and 3D velocity of the particles. Here we omit the robot arm's link structure because arm movements can be derived from end-effector trajectory by inverse kinematies methods. The action space A E IR is then defined as the dD pose change of the end-effectors. For locomotion tasks, the state S = SA considers only the robot agent since the focus is the robot's movement.
Observation. To facilitate learning and optimization algorithms, we employ stratified sampling to avoid the intractably huge state space. We sample different numbers of particles from the scene according to its grid layer, where more particles from the layers closer to the robot agent are sampled since local states are of more importance for policy learning. We sample Ng = E Ni 200 particles from the scene then stack their positions and velocities form Ng 6 vector part of the observation, where L is the number of hierarchical grid layers in the scene and N1 > N2 ... NL. Then we incorporate the robot agent's 's pose to form the observation of dimension of Ns x 6 N.,
B.2
Task Details
In RI. algorithms, we employ stratified sampling and downsampling from all particles to construct the observation input.
In all manipulation tasks, utilize simulation step of 2e-4 seconds, and establish a maximum action range specific to each task to ensure system stability. In locomotion tasks, the simulation step is 2e-3 seconds composed of 10 sub-steps for easier control policy learning. All tasks run faster than real time (more than 60 FPS where each frame is a simulation step) on a laptop computer equipped with an Nvidia RTX 4090 GPU and an Intel i9 CPU.
B.3 Reward Design
In each task, the reward is defined as R -ac B, where L is the total loss of the entire episode, and 3 are task-specific constant for reward scaling.
Sand Painting. We use the state of particles in the ground-truth policy as the target state, compute the chamfer distance d, between the current state and the target state at each time step, and sum them up as the total loss, L = E,de.
Scoop Out. This task has two phases, inserting the shovel into the sand and then lifting the shovel to scoop out the duck. We found that simple matching with goal patterns is hard to learn and not robust enough. Therefore, we divided the learning process into two stages as well. In the first stage, the goal is to encourage the shovel to approach position under the duck, thus 4 = llPehovel Pt ||- where Pt is dynamically computed as Pt = Pdack (0, 0, 0.03). In the second stage, the goal is to encourage the shovel lift and scoop out the duck, thus l2 = -sshonet + Tld,tt where nd, is the number of particles within 0. Im around the duck at time step i. Finally, total loss is computed as L E, 1, (1 o.)) 12), where o, indicates whether step is in the first stage.
Dig A Hole. Similar to the Scoop Out task, this task also consists of two stages. In the first stage, the loss function / = >sshovel =pl encourages the shovel to descend, where zp is the z-coordinate of a predefined point below the sand surface. In the second stage, the loss function L = - -Zshovel 2p) encourages the shovel to ascend. The final total loss is computed the same way as Scoop Out, C E.(a, 4 + (1 (1-o,) o.) 12). where o, indicates whether step is in the first stage.
Smooth Surface. In this task, we use the number of particles above predefined threshold as the metric. Specifically, the loss is defined as L = E 12.t1)), where ng is the number of particles above the average sand surface height.
Quadruped Walk. each step, we compute the distance de between the current state and the target state, represented by the pelvis joint state. The target state is defined as the robot's state that maintains forward velocity close to m/s. The total loss L = E,d, is the sum of these distances.
14

---

Humanoid Stand. To maintain the humanoid robot in standing pose, we compute the delta height between the current state and the initial state, represented by the pelvis joint state dy.e and the head joint state da,t- The total loss L = [epp, + da,,) is the sum of these delta heights.
Humanoid Walk. The target state is defined as the humanoid robot's state that maintains forward velocity close to Im/s without falling to the ground. At each step, compute the distance de between the current and the target pelvis state, reflecting the forward velocity. Additionally, we compute the delta head joint height d representing the standing pose. The total loss L = Ed(d, + da,i) is the sum of the forward distances and the delta heights.
C Additional Experiments Results
Task
Quadruped Sand Walk
Humanoid Sand Stand
Humanoid Sand Walk
PPO
862.8-94.8 -9488
528.3-1:2.11
518.8-32.77
SAC
1572.8 244..
553.1134.7
47..3=34.8
CMA-ES
1037.9=123.3
717.71447.7
524.5:119.3
Task
Quadruped Snow Walk
Humanoid Snow Stand
Humanoid Snow Walk
PPO
2387.4_3361
507.0:112.0
260.2.121.4
SAC
1225.0=201.8
61.
530,9=37.2
CMA-ES
1160.6:155.9
716.61139.9
554.1=30.66
Task
Quadruped Elastic Walk
Humanoid Elastic Stand
Humanoid Elastic Walk
PPO
2387.4_514.6
555.7133666
485.1 332.2
SAC
1214.6=276.6
653.41151.2
579.7=37.3
CMA-ES
1224.4=156.5
686.2:1:31.9
648.8=40.4
Table 3: The final accumulated reward and the standard deviation for each method.
C.1 Locomotion Task Results More Materials
To further investigate our environment, conduct experiments with locomotion tasks on more materials, including sand particles, snow particles, and clastic particles. The results are presented in Table 3 and Figure 8. Similar to the results reported in the main paper, reinforcement learning algorithms generally struggle with these tasks. particular, PPO generally fails on humanoid standing tasks across all surfaces. On the other hand, the sampling-basse trajectory optimization algorithm, represented by CMA-ES, typically achieves better performance. This pattern is further amplified by the large and complex state and action spaces associated with locomotion tasks. Additionally, maintaining a specified pose moving forward consistently remains challenging for end-to-end training, especially on diverse rough surfaces. Consequently, developing an improved policy for locomotion tasks remains a significant challenge.
C.2 Sim-to-Real Transfer
As depicted in Fig. 9, the robotic arm successfully writes the letters "CoRL" in the sand, and scoops the cube from the sand, proving the successful open-loop transfer from UBSoft simulator, which indicates the realistic and potential of our simulator with small sim-to-real gap.
15

---

Quadruped
Humanold Stand
Humanold
137
539
011
0.00
0.00
3.0
M
Extodd
Quadruped Walk
Humanold Stand Snow
Humanoid Snow
3.49
4.53
4.3
691
14.
192
1797
1991
2002
230
730
1326
1966
1740
2010
230
2016
1968
THE
Quadruped Walk Elastic
Humanold Stand Elastic
Humanold Walk Elastic
2120
2000
1000
000
591
Eroods
Ba acol
Spood:
Figure 8: Reward curves for all methods on locomotion tasks, including PPO, SAC, and CMA-ES.
Figure Sand painting and scooping task rollouts in the real world using an XArm.
16


Reviews:
[{"review_id": "JHcmpeGCyw", "reviewer": "Reviewer_gbiB", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 2, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper addresses the problem of simulating soft materials in unbounded regions, as opposed to previous methods that simulate bounded areas because of computational challenges. The authors propose adapting the resolution scales (of the grids and particles) given the robot’s location to adapt MPM [1] based solutions that use fixed grid and particle sizes. Furthermore, 4 manipulation and 4 locomotion benchmark tasks on sand based environments are presented. Different task algorithms are compared with task reward metrics. A qualitative real world study is also provided.", "summary_of_recommendation": "Although the paper has a clear conceptual contribution and show’s effective results to the claims made, the paper lacks experiments with other baseline methods as mentioned in the detailed review. The reviewer also found the technical contribution to be minor. If more experiments are shown in comparison to MLS-MPM and questions/comments are addressed, the reviewer would be more positive about the paper.", "technical_quality": 3, "clarity_of_presentation": 4, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "Comments:\n\nAdditional details on how the physics evolves when the grid sizes/particle sizes change would make the contribution more concrete. \n\nQuestions\n\nDoes the simulation efficiency section compare MPMSA with MPM or MLS-MPM? If MPM, how does the method fair in comparison to MLS-MPM?\n\nFrom the real experiments, is this sim2real or new policies are trained on the real robot? What does it mean for “exhibit trends similar to those observed in UBSOFT”? The reviewer thinks this does not have any backing quantitatively.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HpTJPiZIwG", "reviewer": "Reviewer_P9RJ", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 2, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper proposes a differentiable simulator based on Taichi for simulating large-scale soft materials, introducing a spatially adaptive simulation technique that adjusts resolution based on proximity to the robot agent.", "summary_of_recommendation": "To strengthen the paper, comparative experiments with existing simulators and a clearer distinction of the novelty are needed. Additionally, evaluating the realism of the simulation, expanding related work, and providing detailed experimental clarifications will better demonstrate the proposed method's advantages.", "technical_quality": 2, "clarity_of_presentation": 3, "potential_impact": 2, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "I think the paper has a lot of rooms of improvement in terms of experiments: \n\n1. Comparison with Existing Simulators:\nNeed experiments and metrics to answer, how does the proposed method compare with existing simulators for deformable materials.\n\n2. Distinction and Novelty:\nWhat is the distinction of the proposed simulation from Taichi? Is the spatially adaptive module the primary novelty? If so, can you present experiments illustrating its significance, such as speed-up measurements?\n\n3. Realism and Design Choices:\nHow do you measure the realism of the simulation compared to the real world? Do any design choices hinder the realism of the simulation?\n\n4. Baseline Comparisons:\n- Can you provide an ablation study comparing more agents that do not depend on the proposed simulator (e.g., policies trained using different simulators)?\n\n\nAlso could you clarify:\n\n5. Related Work and Novelty Clarification:\n- Can you elaborate more on related work to clarify the novelty of your approach? What problems do existing SOTA simulators face, and how does your method address them? Is there any literature employing similar ideas to the spatially adaptive module, or is it completely new?\n\n6. Experimental Setup and Task Clarification:\n- Could you elaborate on why the proposed tasks in Section 4.2 cannot be solved by simple replay or heuristic agents (and include them in the comparison)?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HCJJw6vcMP", "reviewer": "Reviewer_2B5z", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The paper presents UBSoft, a new simulation platform designed to support unbounded soft environments (sand, snow, etc) for robot skill acquisition. Via spatially adaptive resolution scales, the proposed framework can markedly improve storage and computation efficiency, which enables simulating large-scale scenarios involving soft materials. The authors also establish a set of benchmark tasks, including both locomotion and manipulation tasks, and conduct experiments to evaluate various RL algorithms and trajectory optimization techniques, both gradient-based and sampling based. Finally, the authors conduct real-world experiments to demonstrate the realism of the UBSoft simulator.", "summary_of_recommendation": "I would recommend weak accept since the proposed simulator seems to be quite advanced and useful for the wider community that might want to study robot interaction with soft material.", "technical_quality": 4, "clarity_of_presentation": 4, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "- Can the simulator support many rigid bodies? How would that impact run time? E.g. burying 100 rigid bodies inside the sand dunes.\n- Why is DiffPhys unable to handle gradients for articulated objects?\n- Have you tried to train the RL agents longer, with more episodes? 1 or 2K episodes seem very restrictive.\n- Do you have more quantitative results for the real-world experiments? Do you observe any performance degradation? If so, what kind of degradation?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: # Strengths: 
Multiple reviewers acknowledge the following strengths in the paper 
- the writeup clarity
- the importance of the problem to simulation and robotics research 
- novelty around spatially adaptive simulation, adjusting the resolution of points based on their proximity to the robot agent. 


# Weakness:
Reviewers have also recognized the following weaknesses
- lack of comparison to MLS-MPM[2], MPM[1] or other existing simulators.
- Clarity around its differences wrt to Taichi. 
- experiments do not convincingly demonstrate the advantages of the proposed method compared to existing simulators for deformable materials 
- Realism of the simulation
- Ambiguity around methods and results - "similar policies are implemented from those in UBSoft" and deployed in reality. The reviewer assumes this means no sim2real transfer. There is no transfer from sim2real and no comparison between different simulation strategies to show the clear effectiveness of the method."

# Recommendations:
It is recommended to extend the comparisons and carefully strengthen the claims. Reviews have provided excellent recommendations on how to improve. Listing a few below - 
- Comparison with Existing Simulators: Need experiments and metrics to answer, how the proposed method compares with existing simulators for deformable materials.
- Additional details on how the physics evolves when the grid sizes/particle sizes change would make the contribution more concrete.
- From the real experiments, is this sim2real, or new policies are trained on the real robot? What does it mean to “exhibit trends similar to those observed in UBSOFT”? The reviewer thinks this does not have any backing quantitatively. 
-  What is the distinction of the proposed simulation from Taichi? Is the spatially adaptive module the primary novelty? If so, can you present experiments illustrating its significance, such as speed-up measurements?

# Rebuttal 
Authors and reviewers engaged in healthy discussions during rebuttals. Post rebuttals, all reviewers are aligned on the recommendation of the paper. In support of their feedback, AC recommends accepting the paper. Reviewers should address all the pending commitments from the rebuttals and prepare the final manuscript accounting for all the received feedback.
Decision: Accept