DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation
Authors: Zilin Si
Gu Zhang
Qingwei Ben
Branden Romero
Zhou Xian
Chao Liu
Chuang Gan
Venue: ICLR 2024 poster (2024)
Keywords: Tactile sensing
Simulation
Robotic manipulation
Primary area: applications to robotics, autonomy, planning

Abstract: We introduce DIFFTACTILE, a physics-based differentiable tactile simulation system designed to enhance robotic manipulation with dense and physically accurate tactile feedback. In contrast to prior tactile simulators which primarily focus on manipulating rigid bodies and often rely on simplified approximations to model stress and deformations of materials in contact, DIFFTACTILE emphasizes physics-based contact modeling with high fidelity, supporting simulations of diverse contact modes and interactions with objects possessing a wide range of material properties. Our system incorporates several key components, including a Finite Element Method (FEM)-based soft body model for simulating the sensing elastomer, a multi-material simulator for modeling diverse object types (such as elastic, elastoplastic, cables) under manipulation, a penalty-based contact model for handling contact dynamics. The differentiable nature of our system facilitates gradient-based optimization for both 1) refining physical properties in simulation using real-world data, hence narrowing the sim-to-real gap and 2) efficient learning of tactile-assisted grasping and contact-rich manipulation skills. Additionally, we introduce a method to infer the optical response of our tactile sensor to contact using an efficient pixel-based neural module. We anticipate that DIFFTACTILE will serve as a useful platform for studying contact-rich manipulations, leveraging the benefits of dense tactile feedback and differentiable physics. Code and supplementary materials are available at the project website https://difftactile.github.io/.

Published as a conference paper at ICLR 2024
DIFFTACTILE: A PHYSICS-BASED DIFFERENTIABLE TACTILE SIMULATOR FOR CONTACT-RICH ROBOTIC MANIPULATION
Zilin Si % t, 1.5, Gu Zingei Ben +3 Branden Romero", Zhou Xian',
Chao Liu Chuang Gan
1 CMU RI, 2 Shanghai Jiao Tong University, Tsinghua University, MIT CSAIL.
5 MIT-IBM Watson AI Lab, UMass Amherst
zsi @andrew. cmu, edu, guzhang sjtu@gmail...... com,
bqw2Demails. wwwpaailsssssingaaaaaaa wdw.inguua.euuu.... edu. en, brromero@mit. xianzl@andrew cmu edu, chaoliu@csail wttp:saaa..mmmteii.... edu, ganchuang@csail wtt.ssaaa...meeeemmm.e...m....ee mittedu
ABSTRACT
We introduce DIFFTACTILE, a physics-based differentiable tactile simulation system designed to enhance robotic manipulation with dense and physically accurate tactile feedback. In contrast to prior tactile simulators which primarily focus on manipulating rigid bodies and often rely on simplified approximations model stress and deformations of materials in contact, DIFFTACTILE emphasizes physics-based contact modeling with high fidelity, supporting simulations diverse contact modes and interactions with objects possessing wide range of material properties. Our system incorporates several key components, including a Finite Element Method (FEM)-based soft body model for simulating the sensing elastomer, multi-material simulator for modeling diverse object types (such elastic, elastoplastic, cables) under manipulation, penalty-based contact model for handling contact dynamics. The differentiable nature of our system facilitates gradient-based optimization for both refining physical properties in simulation using real-world data, hence narrowing the sim-to-real gap and 2) efficient learning of aastile-assiste grasping and contact-rich manipulation skills. Additionally, we introduce method to infer the optical response of our tactile sensor to contact using an efficient pixel-based neural module. We anticipate that DIFFTACTILE will serve as useful platform for studying contact-rich manipulations, leveraging the benefits of dense tactile feedback and differentiable physics. Code and supplementary materials are available at the project website'.
INTRODUCTION
In the goal of enabling robots to perform human-level manipulation diverse set of tasks, touch is one of the most prominent components. Tactile sensing, as a modality, unique in the sense that provides accurate, fine-detailed information about environmental interactions in the form of contact geometries and forces. Although its efficacy has been highlighted by prior research, providing crucial feedback in grasping fragile objects (Ishikawa et al., 2022), enabling robots to perform in occluded environment (Yu & Rodriguez, 2018), and detecting incipient slip (Chen et al., 2018) for highly reactive grasping, there are still advances in tactile sensing to be made especially in the form of simulation.
baseed simulation has become significant practical tool in the domain of robotics, by mitigating the challenges of real-world design and verification of learning algorithms. However, existing robotic simulators either lack simulation for tactile sensing or limit interactions to rigid
AAuthors with equal contribution.
I This work was done during an internship at the MIT-IBM Watson AI Lah. httpp: / //i fftact 1e.gi thub. 1

---

Published as a conference paper at ICLR 2024
bodies. To accurately simulate tactile sensors which inherently soft, it is essential to model soft body interaction's 's contact geometries, forces, and dynamics. Prior work (Si & Yuan, 2022) attempted to simulate contact geometries and forces for tactile sensors under (quasi-) Jstatic scenarios, and it was successfully applied to robotic perception tasks such as object shape estimation (Suresh al., 2022), and grasp stability prediction (Si et al., 2022). However, highly dynamic manipulation tasks have not been thoroughly explored. Other prior works approach contact dynamics by either approximating sensor surface deformation using rigid-body dynamics (Xu et al., 2023) or using physics-based soft-body simulation methods such as Finite Element Method (FEM) (Narang et al., 2021). However, these methods are still limited manipulating rigid objects.
In this work, we aim to build a differentiable tactile simulator, DIFFTACTILE supports contact-rich robotic manipulation of rigid, deformable, and articulated objects. Differentiabilityy as key component of our work, provides fine-grained guidance for efficient skill learning (Huang et al., 2021; Xian et al., 2022). It also enables system identification to close the sim-to-real gap (Li et al., 2023). We implement DIFFTACTILE in Taichi (Hu et al., 2019) which leverages parallel GPU computing and auto-differentiation. To demonstrate the capability and versatility of our simulator, we evaluate it on diverse set of manipulation tasks including handling fragile, deformable, dynamic objects that cannot be addressed with prior tactile simulators. We summarize our contributions below:
that
Figure 1: Grasping a deformable object in the real world and in DIFTTACTIE.
We introduce DIFFTACTILE platform supporting various tactile- assisted manipulation tasks. We model tactile sensors with FEM, objects various materials (rigid, elastic, and elastoplastic) with Moving Least Square Material Point Method (MLS-MPM), and cable with Position-Bast Based Dynamics (PBD). We simulate the contact between sensors and objects with a penalty-bat -assdd contact model. In addition, we accurately simulate the optical response of tactile sensors with high spatial variation via learning- method.
Our system is differentiable and can reduce the sim-to real gap with system identification. From sequence of real-world data samples, we can optimize our simulator's sensor material and contact model parameters with differential physics and validate it with more general real-world scenarios.
We demonstrate the improvement of skill learning efficiency with tactile feedback. We evaluate it on stable and adaptive grasps of objects with diverse geometry and material properties, and four contact-rich manipulation tasks.
RELATED WORK
Tactile simulation The most recent work on tactile simulation is built upon existing rigid-body simulators. For example, Tacto (Wang et al., 2022), Tactile-Gym (Church et al., 2022; Lin et al., 2022) were built upon PyBullet (Coumans & Bai, 2016). An efficient tactile simulation (Xu al., 2023) was built upon DiffRedMax (Xu et al., 2021), where penalty-based contact model was used to simulate the force distribution for tactile sensors. Even though computationally efficient to use rigid body simulation, these tactile simulators approximate contact dynamics for soft bodies at the cost fidelity.
Alternatively, Finite Element Method (Fed)-based methods exist to accurately simulate soft body dynamics. A physies-based tactile simulator (Narang et al., 2021) was developed for SynToocc BioTac sensors (SynTouch) by using FEM in Isaac Gym (Makoviychuk simulator also used the FEM in Isaac Gym (Kim et al., 2022) with incremental potential contact (IPC) model handle contact dynamics. Taxim (Si & Yuan, 2022) used superposition method to approximate the FEM. We also model tactile sensors with FEM to maintain the simulator's 's physical accuracy and extend the contact model handle objects with various materials beyond rigid.
al., 2021). A grasp
Differentiable physics- based simulation Differentiable physics-basee based simulation has become popular in recent years it allows for efficient gradient-based policy learning compared to
2

---

Published as a conference paper at ICLR 2024
Object model
Backend
Method
Optical
Differentiability
Tactile simulator
Rigid
Soft
Simulation
Tacto (Wang al., 2022)
PyBullet
Rigid body
(Xu al., 2023)
DiffRedMax
Rigid body MPM
Tacchi (Chen et al., 2023)
Taichi
Taxim (Si &
PyBullet
FEM
Yuan, 2022)
(Narang et al., 2021)
Isaac Gym
FEM
IPC-
Isaac Gym
FEM
GraspSim (Kim al., 2022)
Ours
Taichi
FEM
Table I: Comparison with other state-of-the thr tactile simulators. We show that DIFFTACTILE is the only tactile simulator supporting simulating objects with various materials while being system-wis differentiable physically accurate.
traditional sampling-ben based algorithms. PlasticineLab (Huang et al., 2021), FluidLab (Xian et al., 2022), SoftZoo (Wang et al., 2023) were presented with differentiability for soft body manipulation, fluid manipulation, and soft robot co-design, respectively, by leveraging Moving Least Square Material Point Method (MLSSMPMM) (Hu al., 2018). Tacchi (Chen al., 2023) also used MIS- MPM to simulate the soft body deformation for GelSight (Yuan et al., 2017), a type of vision- based tactile sensor but did not present differentiability and contact dynamics modeling. It is shown that differential physics can be applied for system identification (Ma et al., 2023) to fine- tune the simulator's physical parameters and reduce the sim-to-real gaps. However, it remains unclear whether the gradient-based approach can benefit to improve the efficiency of tactile-assisted manipulation skill learning.
Optical Simulation Taxim (Si & Yuan, 2022) showed that data-driven approaches to simulate the optical response of vision-based tactile sensors significantly outperform model-based methods such as (Wang et al., 2022; Chen et al., 2023; Agarwal et 2021; Gomes et al., 2021). However, there is a divergence data-driven approaches. Previous work including (Higuera et al., 2023; Chen al., 2022; Zhong et al., 2023) use image generation techniques like generative models to perform style transfer from simulated image to the style a real deformation. However, these methods are rather data-intensive since they need a large variation of real-world examples generalize well. Instead, Taxim (Si & Yuan, 2022) takes a pixel- based approach that uses a polynomial lookup table to map surface normals to RGB directly. It is data-efficient but makes assumptions about the sensors bidirectional reflectance distribution function (BRDF), which limits its applicability to sensors with low spatial variance.
We compare our work with state-of-the-art tactile simulators in Table 1. We show that our work, to the best of our knowledge, is the only work that is 1) system-wise differentiable enable efficient skill learning, 2) can accurately model the soft body dynamics and contact dynamics, 3) supports broad categories of objects including rigid, elastic, elastoplastic, and cables, and 4) provide data- efficient approach to simulate optical responses for vision-based tactile sensors.
TACTILE SIMULATION
3.1 SYSTEM OVERVIEW
DIFFTACTILE models the soft contact between tactile sensors and objects including contact force distribution, contact surface deformation, and optical response to provide dense tactile feedback. We present four key modules of our system: 1) a Finite Element Method (FEM)-based tactile sensor model in Section 3.2, 2) a learning- based method to simulate the optical response of tactile sensors with high spatial variation in Section 3.3, 3) rigid, elastic, and elastoplastic object models using Moving Least Square Material Point Method (MLS-MPM), and cable model using Position-Based Dynamics (PBD) in Section 3.4, 4) penalty-base based contact model in Section 3.5.
3

---

Published as a conference paper at ICLR 2024
3.2 TACTILE SENSOR SIMULATION
We model the deformation of the tactile sensor's soft elastomer under contact forces with FEM. We discretize the sensor soft elastomer tetrahedron elements and then apply boundary conditions at the base of the sensor with position or velocity control. Since most tactile sensors' clastomers including ours are made from hyper-elastic materials, we apply the Neo-Hookean constitutive model in our simulation to capture the non-linearity of the material property. The energy density function V and the first Kirchhoff stress tensor P used for governing equations are defined as:
(1,, J)
2 14 (11 3) (]log(]) + 12++22100 (J)
(I)
P(F) = 1(F F F) + \log(J)F
where F E R3x3 is the deformation gradient, 1 F) is the first isotropic invariants, and = dFt(F) is an additional invariant. Note that our tactile simulation can be easily customized with different shapes, sizes, and materials by replacing the input mesh model constitutive model.
To get tactile outputs including visual images and marker motions for vision-basee tactile sensors, first extract the deformed surface mesh from each simulation step's FEM solution, then interpolate the marker's 's locations by weighting surface node locations given a set of initial markers captured from a real sensor. We project 3D markers to the 2D image plane by using the tactile sensor's s camera model.
3.3 OPTICAL SIMULATION
We reconstruct the optical response of vision-based tactile sensor to contact using a data-driven approach. We model the surface of the sensor as a height function = [xx 3). and represent the continuous spatially-varying reflectance function of the surface as 4D vector-valued function. The Of Øf function input is the 2D viewing direction (d 0, (p) and 2D surface normals (x 8 ), and g.b..
the output is the change in reflected color with a multilayer perceptron (MLP) fo whose input is augmented with a positional encoding (d) and (xx) rather than directly d and enable the network to better fit data with high-frequency variation (Mildenhall et al., 2021). Formally the encoding function is:
b). We approximate reflectance function
7p)) sin(22Tpp) cos(22
sin(2"-'Tp)) cos(2"-)mmp)
Our rendering scheme finally consists of approximating the deformation caused by the contact indentation using pyramid Gaussian kernels as proposed in (Si & Yuan, 2022).
3.4 OBJECT SIMULATION
We aim support broader categories of objects beyond rigid objects for more diverse manipulation applications. We leverage the Moving Least Square Material Point Method (MLS-MPM) (Hu et al., 2018) to simulate rigid, elastic, elastoplastic objects. MLS-MPM has been shown be efficient in simulating soft bodies. For elastic objects, we implement both corotated linear elasticity and Neo- Hookean elasticity models. For elastoplastic objects, we use the von Mises yield criterion to model plasticity upon elasticity. For rigid objects, we first treat objects as elastic using MLS-MPM, and then we add rigidity constraints by calculating object transformation and enforcing the shape of the object. For articulated objects, and assign different materials for different parts. The joints are simulated as soft and thin bodies and other parts are simulated rigid bodies.
approximate the simulation by using the MPM-based approach
For another group of deformable objects such as cables and clothes, it is common to simulate them with Position Based Dynamics (PBD) (Müller al., 2007). We also incorporate cable objects in our simulation by using PBD, where we constrain the stretch, bending, and self-collision.
3.5 PENALTY-BASED CONTACT MODEL
We handle contact dynamics between sensors and objects with a penalty-basee based contact model similar to (Xu et al., 2023). At each simulation step, we first check contact collision by pairing the surface
4

---

Published as a conference paper at ICLR 2024
triangle mesh from FEM with surface nodes from the object's particles (with either MPM or PBD). For each pair, we calculate the sign distance field d and normal directions n from the node to the triangle mesh. If d is negative, the node is penetrating the surface mesh and we need to apply normal penalty force to both mesh nodes and particle node to constrain the contact. In addition, we apply static dynamic friction forces to the pair based their relative velocities and normal forces. We represent our contact model as:
fn = -kip + kav
f. = Vt min(kellvell. ((ll))
livell
where fn and f; are contact forces in the normal and tangential direction with respect to the local surface triangle. Vn and Vt the relative velocities between the pair of the triangle and node in normal and tangential directions. kn, ka, k; and fr are the parameters of contact stiffness, contact damping, friction stiffness, and friction coefficient. Then the contact force fn is applied to both the triangle mesh nodes and the particle node of the pair as an external force.
FEM-MPM coupling FEM is mesh-based method and we can extract surface triangle meshes along with their associated node positions, velocities and face normal directions. MLS-MPM is meshless hybrid Lagrangian- Eulerian method that uses Lagrangian particles and Eulerian grids to simulate continuous materials. We apply contact collision checking and contact force modeling between FEM surface mesh nodes and MPM Eulerian grids for efficiency.
In each simulation step, we first pre-compute the internal elastic forces for all tetrahedral meshes from the constitutive law for the FEM sensor model, and advance particles to grids for the MPM object model. Then we check contact collision and calculate external contact forces for all pairs of triangle meshes and grid, and add them to the surface nodes. post-contact computing, we transfer the velocities and affine coefficients from the grid to particles and do particle advection for the MPM object model; and we advect the positions and velocities of the nodes based on the internal elastic forces, external contact forces, and gravity for FEM elements. We also consider the external boundaries such as tables and walls to constrain the positions of the objects.
FEM-PBD coupling Similarly FEM-MPM coupling, simply replace the MPM particles with PBD particles for contact collision detection and modeling. For PBD objects, there's s no pre- contact computation, but need to solve the stretch, bending, and self-collision constrains after the contact, and velocity advection based on the updated positions.
4 EXPERIMENTS
4.1 OVERVIEW
We present two sets of tasks with DIFFTACTILE: system identification, and tactile-assisted manipulation. For system identification, we use real-world tactile observations optimize the simulator's ' system parameters and to reduce sim-to-real gaps. Then we present five manipulation tasks: grasping, surface following, cable straightening, case opening, and object reposing as shown in Fig. 2. Tactile sensing can enable safer and more adaptive grasping to handle fragile objects such as fruits. We grasp a diverse set of objects with various shapes, sizes, and materials without slipping and damaging. For the other four contact-rich manipulation tasks, surface following requires the sensor to stay in contact with a 3D surface and travel to an endpoint while maintaining a certain contact force; cable straightening requires a pair of sensors to first grasp fixed end of the cable, and then straighten it by sliding towards the other end; case opening uses a single sensor to open an articulated object via pushing; lastly, object reposing involves using single sensor to push object from a lying pose to a standing pose against the wall. These four tasks represent rigid, deformable, and articulated object manipulation.
4.2 SIMULATION SETUP
Initialzzation We initialize the simulation environment with single tactile sensor for system identification, surface following, case opening, and object reposing, and two tactile sensors {81, $2) mounted parallel jaw gripper for grasping and cable straightening. Both tactile sensors' and
5

---

Published as a conference paper at ICLR 2024
Task Initial
0bj set
Task Final
Tactile view
Grasping
Surface fo Toolowg
Cable
Object reposing
Case opening
Straightening
grasp set four objects with different geometries and materials. Surface following: A sensor travels on the surface while maintaining the contact. Cable straightening: A pair of sensors straighten cable by gripping and sliding from fixed end. Object reposing: sensor pushes object let it stand against wall. Case opening: A sensor opens the cap of a case.
Figure 2: DIFFTACTILE tasks. Grasping:
objects' shapes initialized with STL or OBJ mesh models and then voxelized FEM tetrahedron meshes or MPM/PBD particles. Objects o, are initialized statically on the tabletop and we add a vertical wall for object reposing. Tactile sensors
initialized statically near objects depending on tasks but without contact. We initialize the poses of tactile sensor at time step & = 0 as (.( (O), (0)) E SE(3) where R2 E SO(3) and (0) E R and similarly object pose (0).
State Each tactile sensor is represented as an FEM entity with N nodes and M tetrahedral elements. For each node n4, contacts 6D state vector (t) (t), v,(t)} including 3D position (t) and 3D velocity (t). For each element m, it contacts 4D index mapping from the element its associated four nodes. Both MPM- based and PBD-based objects are represented with particles, and similarly, each particle also has a 6D state vector (t (p)(t), (t)).
Observation We define two types of observations of each simulation step f, the state observation and the tactile observation. State observation includes tactile sensors' and objects' poses T, ((),), T(()) and each node's or particle's state (1))) (t). For tactile observation, we can output the sensor's surface triangle mesh as deformation map, the sensor's surface force distribution, threc-axis force vector.
aggregated
Action At each time step t, actions for end-effectors (either tactile sensors gripper with kinematic chains down to tactile sensors) are queried from the controller as represented as velocity vector (t) (AR,(t), (t) } to update the velocities of the FEM nodes.
Reward/Loss Each task's reward or loss function is formed differently based on the task objectives. We refer the readers to Section A.4 for more details.
4.3 SYSTEM IDENTIFICATION
Sim-to-real transfer for robot learning has been long-standing challenge where the gap in between heavily relies on simulation fidelity. To reduce the gap, we leverage differentiable physics to optimize the physical parameters of material and contact models given example data from the real world. Our optimization targets include Lamé parameters μ and of the FEM sensor model, and kn, ka, kt, l of the contact model. The optimization objectives include the 6-axis force readings and tactile marker readings under four different contact scenarios: pressing, sliding, in-plane twisting, and tilt twisting shown in Fig.
Experimental Setup and Dataset We collect sequences of contact data from both the real world and simulation with synchronized control poses and velocities of the sensor.

---

Published as a conference paper at ICLR 2024
Press-slide),
Press-twist-z,
Press-twist-x,
Random
1.69 + 1.10
1.15 0.51
1.43 + 0.62
RNN
1,20 0.42
0.68 + 0.28
0.90 + 0.26
Sim2Sim
CMA-ES
0.59 0.12
0.47 + 0.15
0.61 + 0.20
Ours
0.53 0.35
0.42 0.24
0.58 0.28
Random
3.54 1.73
2.59 0.99
4.47 3.31
RNN
3.29 + 1.51
2.42 0.90
Real2Sim
CMA-ES
3.42 + 1.47
2.67 + 1.22
4.99 4.30
Ours
3.08 + 1.27
2.38 + 0.86
3.99 2.89
Table 2: The pixel-wise tactile marker mean squared emrors with standard deviation to evaluate system identification.
As shown in Fig. 3, there are three types of data sequences, press-slide, press-hvist-z (twist along the z-axis), and press-twist-x (twist along the x-axis). For this experiment, the sensor interacts with two surfaces with different frictional properties, acrylic and tape. Experimental results
We External
Material
evaluate two sets of experiments: Sim2Sim and Real2Sim where use simulated data real data respectively as inputs of the system. We optimize the sensor and contact model parameters with press-slide all three
Tactile
Sirplated
Acrylic
sequence and test sequences. We compare gradient- based trajectory optimization (Ours) with three baselines, Random, RNN, and CMA-ES shown in Table 2. Here we use pixel-wise mean squared error (MSE) between predicted and collected tactile markers as evaluation metrics. For Random, we randomly select parameters within a practical range; for RNN, input tactile marker readings and force readings and output the predicted system parameters; for CMA-ES, we sample predicted parameters from algorithm's 's distribution function. Ours outperforms Random, RNN and CMA-ES all sequences for both Sim2Sim and Real2Sim.
Press
Slide
Twist
Tilt
Tape
Figure System identification to optimize the FEM sensor model and contact model's physical parameters with tactile readings and force readings from the real world.
We use the identified tactile sensor parameters from Real2Sim for all following manipulation tasks. However, contact parameters such as the surface friction coefficient also depend on object materials. But these can still serve as good references and we use them by adding randomization based on the identified parameters. For object parameters, we randomize them within range based on the tactile sensor's s and contact model's parameters make sure the system can stably run.
4.4 OPTICAL SIMULATION
Experimental setup and dataset We manually collect 250 example deformations across the entire sensing surface using spherical indenter from the real world. The pose of the sphere manually annotated, and we split the dataset into a training set consisting of 200 examples, with the rest held out for testing.
Reat
4mm
Sinutated
Experimental results We test method against a polynomial table mapping from Taxim (Si & Yuan,
Figure 4: Tactile optical simulation compared with real data capturing various contact geometries.

---

Published as a conference paper at ICLR 2024
L14
MSE.
SSIM
PSNR
Taxim
16.1
85.74
0.998
38.47
Ours
7.94
56.10
0.999
39.42
Table 3: Image similarity metries for our test We compare our method Taxim (Si Yuan, 2022) on LI,
MSE, SSIM and PSNR metrics. Our method performs better across all metrics.
Loss
Lpon
Daipp+
Laeform
w/o tactile
0.04 + 0.03
0.18 + 0.06
N/A
Elastic
tactile
0.01 0.01
0.07 0.04
Elasto-
w/o tactile
0.70 + 0.53
0.26 + 0.09
0.85
0.25
plastic
tactile
0.24 0.01
0.2 0.05
0.89 0.48
Table 4: Evaluation of grasping deformable, fragile objects with position, deformation losses, and slipping distance by either using or not using tactile observations.
2022). We use pixel-wise MSE, LI, SSIM, and PSNR as evaluation metrics. As shown in Table 3, method outperforms Taxim across all metrics. Additionally, accuracy of our method by rendering a set of test probes with varying geometry, along with example seal-world indentations for comparison in Fig. 4. We show method can capture contact geometries in great detail.
verify the generalization and
4.5 GRASPING
Experimental setup and dataset We evaluate our simulator on grasping objects with various object properties including different shapes, sizes, weights, and material properties. As shown in Fig. 2, we select four objects from EGAD (Morrison al., 2020) dataset with different shape complexity and assign each object with two different material properties, elastic, and elastoplastic.
We aim to grasp objects stably and adaptively avoid slipping and damaging the object with gradient-based trajectory optimization. Here we use two tactile sensors as fingertips and mount them on parallel jaw gripper. In each trajectory, the gripper first grips the object and then lifts it. Based on goal, define the objectives with three types of losses 1) Position loss Lpos: we set a 3D target position to reach after lifting; 2) Deformation loss form: we aim to keep the shape of the object during the grasp by using the sign distance field of the object and the L1 distance of the mass distribution between the current object and the target one to penalize the deformation (Huang al., 2021) Slipping loss Latipi we use the shear force detected between the fingertip and the object to penalize the slippage during grasping.
Experimental results We evaluate the grasping with or without tactile feedback three metries. We use Lyos for both types of objects, and we use form for elastoplastic objects only. In addition, measure the slipping distance of the object relative to the sensor for both sets of objects, the slipping distance is denoted as D Dalig- We show in Table 4 that the tactile feedback greatly improves the grasping quality.
4.6 CONTACT-RICH MANIPULATION
Experimental setup For all four manipulation tasks, we define two different rewards, state reward and tactile reward for manipulation skill learning. We evaluate our system's 's learning efficiency by comparing gradient-based trajectory optimization with sampling-based trajectory optimization, CMA-ES (Hansen et al., 2003), and model-free RL algorithms, SAC (Haarnoja et al., 2018), and PPO Schulman et al. (2017).
Surface following We sct up a sensor to travel and follow a curved 3D surface. We define the state reward traveling to certain position the 3D surface, and the tactile reward as keeping contact with the surface while maintaining a constant shear motion.
Cable straightening We set up parallel jaw gripper with two tactile fingers and cable with one end fixed to the wall while the other end is free. The state reward is defined as the distance between the target position (the cable is horizontally straight) and the current position for each node the
8

---

Published as a conference paper at ICLR 2024
Obs
Rew
Manipulation tasks
w/ tac
w/
ObjectRepose
CableStraighthen.
CaseOpen
SurfaceFollow
X
4.57 = 0.06
2.06 0.00
-0.95 + 0.04
1.33 1.53
X
1.49 + 0.08
2.07 + 0.02
-0.93 + 0.26
1.67 + 0.58
PPO
4.64 + 0.15
2.03 + 0.04
-0.83 + 0.06
2.67 + 1.15
4.30 = 0.15
1.90 0.18
-0.80 = 0.24
1.33 1.53
5.00 0.01
1.50 0.02
-0.68 0.29
11.00 0.00
4.90 = 0.01
2.03 0.02
-0.89 + 0.09
10.00 + 5.57
SAC
X
4.89 + 0.11
1.60 + 0.12
-0.84 + 0.04
14.00 + 2.00
4.68 + 0.11
1.36 + 0.03
-0.95 + 0.07
1.33 + 1.53
CMA- ES
N/A
4.65 + 0.14
1.97 0.14
-1.07 + 0.05
2.33 + 1.15
N/A
4.50 + 0.05
1.97 0.15
-0.98 + 0.07
1.67 1.15
N/A
12.07 + 12.46
1.27 + 0.81
17.11 + 0.05
4.00 + 1.00
Ours
N/A
60.82 1 0.00
0.89 +: 0.32
9.83 + 0.38
51.67 1 12.86
Table 5: Evaluation of manipulation tasks by comparing gradient-base based optimization (Ours) with sampling- based optimization (CMA-ES), and reinforcement learning approaches (SAC, PPO).
cable. The tactile reward defined as the force applied the cable maintain the gripping while being able to slide along the cable.
Case opening We initialize closed case and we use tactile sensor to push and open the lid of the case. We define the state reward as the angle of the opened lid and the tactile reward as the push forces to open the lid.
Object reposing A block is placed flat on the table and we aim to use one tactile sensor to flip 90 degrees and make it stand against wall. We define the state reward as the angle between the object and the floor, and the tactile reward the push forces to flip the object.
Experimental Results To evaluate the performance of trained policies for different tasks, we design task-specific evaluation metries: We use the traveling distance of the sensor in contact with the surface for the surface following task; the aggregation distance between the current and target cable nodes' locations for the cable straightening task; the orientation changes of the lid and the object from the beginning to the end the trajectories for case opening and object reposing tasks.
We show all experimental results in Table 5 by comparing our proposed gradient-based optimization method with baselines. We show Ours outperforms baselines with a large margin to show its learning efficiency. And w/ tactile has better performances compared to w/o tactile for most tasks indicating tactile sensing helps on these contact-rich manipulation tasks.
CONCLUSIONS AND FUTURE WORK
We present DIFFTACTILE, physics-based based differentiable tactile simulator to advance skill learning for contact-rich robotic manipulation. By providing models for tactile sensors, multi-material objects, and penalty-based contacts, we greatly extend the capabilities and applicability of robotic simulators. The differentiability of our system aids reducing the sim-to-real gaps by using system identification and improves the skill learning efficiency by providing gradient- based optimization. We evaluate DIFFTACTILE 's versatility with the grasp of set of various objects, and manipulation tasks including surface following, cable straightening. case opening, and object reposing. By comparing with the otate-of-the-art reinforcement learning and sample-based trajectory optimization approaches, demonstrate that DIFFTACTUS sensing and potentially serve as a learning platform for broader tactile-assisted manipulation tasks.
enable efficient skill learning with tactile
In future work, we plan to integrate our tactile simulator into commonly used robotic simulation frameworks to extend its usage on more general manipulation configurations such as adding tactile sensors dexterous robotic hands for in-hand manipulation. We would also like to investigate robot learning with molti-modalities simulation such as leveraging vision and touch feedback to improve the robustness of the policies.

---

Published as a conference paper at ICLR 2024
REFERENCES
Arpit Agarwal, Timothy Man, and Wenzhen Yuan. Simulation of vision-based tactile sensors using physics based rendering. In 2021 IEEE International Conference Robotics and Automation (ICRA), pp. 1-7. IEEE, 2021.
Wei Chen, Heba Khamis, Ingvars Birznieks, Nathan Lepora, and Stephen J. Redmond. Tactile sensors for friction estimation and incipient slip detection toward dexterous robotic manipulation: A review. IEEE Sensors Journal, 18(22):9049-9066 9064, 2018. doi: 10.1109/JSEN. 2018.2868340.
Weihang Chen, Yuan Xu, Zhenyang Chen, Peiyu Zeng, Renjun Dang. Rui Chen, and Jing Xu. Bidirectional sim-to-real transfer for gelsight tactile sensors with cyclegan. IEEE Robotics and Automation Letters, (3)):6177-6194) 2022.
Zixi Chen, Shixin Zhang, Shan Luo, Fuchun Sun, and Bin Fang. Tacchi: A pluggable and low computational cost elastomer deformation simulator for optical tactile sensors. IEEE Roboties and Automation Letters, 8(3): 1239:1229-146, 2023.
Alex Church, John Lloyd, Raia Hadsell, and Nathan F. Lepora. Tactile sim-to-real policy transfer via real-to-sim image translation. In Aleksandra Faust, David Hsu, and Gerhard Neumann (eds.), Proceedings of the 5th Conference on Robot Learning, volume 164 of Proceedings of Machine Learning Research. PMLR, 08-11 Nov 2022. URL URR v164 64/ hhwprch222222 html.
mll. press/
Erwin Coumans and Yunfei Bai. Pybullet, a python module for physics simulation for games, robotics and machine learning. 2016.
Daniel Fernandes Gomes, Paolo Paoletti, and Shan Luo. Generation of gelsight tactile images for sim2real learning. IEEE Robotics and Automation Letters, 6(2):4177 2021.
Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pp. 1861-1870. PMLR, 2018.
Nikolaus Hansen, Sibylle D Müller, and Petros Koumoutsakos. Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (cma-es). Evolutionary computation, 11(1):1-18, 2003.
Carolina Higuera, Byron Boots, and Mustafa Mukadam. Learning read braille: Bridging the tactile reality gap with diffusion models. arXiv preprint arXiv:2304.01 182, 2023.
Yuanming Hu, Yu Fang, Ziheng Ge, Ziyin Qu, Yixin Zhu, Andre Pradhana, and Chenfanfu Jiang. A moving least squares material point method with displacement discontinuity and two-way rigid body coupling. ACM Transactions on Graphics (TOG), 37(4 2018.
Yuanming Hu, Luke Anderson, Tzu-Mao Li, Qi Sun, Nathan Carr, Jonathan Ragan-Kelles and Fredo Durand. Difftaichi: Differentiable programming for physical simulation. In International Conference on Learning Representations, 2019.
Zhiao Huang, Yuanming Hu, Tao Du, Siyuan Zhou, Hao Su, Joshua B Tenenbaum, and Chuang Gan. Plasticinelab: A soft-body manipulation benchmark with differentiable physics. arXiv preprint arXivv2104.03311, 2021.
Reina Ishikawa, Masashi Hamaya, Felix Von Drigalski, Kazutoshi Tanaka, and Atsushi Hashimoto. Learning by breaking: food fracture anticipation for robotic food manipulation. IEEE Access, 10: 99321-99329 2022.
Chung Min Kim, Michael Danielczuk, Isabella Huang, and Ken Goldberg. Ipc-graspsim: Reducing the sim2real gap for parallel-jaw grasping with the incremental potential contact model. In 2022 International Conference on Robotics and Automation (ICRA), pp. 6180-6187. IEEE, 2022.
Xuan Li, Yi-Ling Qiao, Peter Yichen Chen, Krishna Murthy Jatavallabhula. Ming Lin, Chenfanfu Jiang, and Chuang Gan. Pac-nerf: Physics augmented continuum neural radiance fields for ecometry-agnostee system identification. arXiv preprint arXiv:2303.055111 2023.
10

---

Published as a conference paper at ICLR 2024
Yijiong Lin, John Lloyd, Alex Church, and Nathan F. Lepora. Tactile gym 2.0: Sim-to-real deep reinforcement learning for comparing low-cost high-resolution robot touch. volume of Proceedings of Machine Learning Research, pp. 10754-10761. IEEE, August 2022. doi: 10. 10.1109/1RA.202.331999
Pingchuan Ma, Peter Yichen Chen, Bolei Deng. Joshua B Tenenbaum, Tao Du, Chuang Gan, and Wojciech Matusik. Learning neural constitutive laws from motion observations for generalizable pde dynamics. arXiv preprint arXiv:2304. /4369, 2023.
Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, et al. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2 10470, 2021.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99- 106, 2021.
Douglas Morrison, Peter Corke, and Jürgen Leitner. Egad! an evolved grasping analysis dataset for diversity and reproducibility in robotic manipulation. IEEE Robotics and Automation Letters, (3):4368-4375, 2020.
Matthias Müller, Bruno Heidelberger, Mareus Hemnix, and John Ratcliff. Position based dynamics. Journal of Visual Communication and Image Representation, 18(2):109-118, 2007.
Yashraj Narang, Balakumar Sundaralingam, Miles Macklin, Arsalan Mousavian, and Dieter Fox. Sim-to-real for robotic tactile sensing via physics-based simulation and learned latent projections. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 6444-6451. IEEE, 2021.
Antonin Raflin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. Stable-baselines:: Reliable reinforcement learning implementations. The Journal of Machine Learning Research, 22(1):1234 - 2021.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
Zilin Si and Wenzhen Yuan. Taxim: An example-based simulation model for gelsight tactile sensors. IEEE Robotics and Automation Letters, 7(2):2361-2368, 2022.
Zilin Si, Zirui Zhu, Arpit Agarwal, Stuart Anderson, and Wenzhen Yuan. Grasp stability prediction with sim-to-real transfer from tactile sensing. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 7809-7816, 2022. 9981863.
doi: 10. 10.110//R0047612.2222.
Sudharshan Suresh, Zilin Si, Joshua G Mangelson, Wenzhen Yuan, and Michael Kaess. Shapemap 3-d: Efficient shape mapping through dense touch and vision. In 2022 International Conference on Robotics and Automation (ICRA), pp. 7073-7080. IEEE, 2022.
SynTouch. https: http:::://yntttchinnc.. com.
Shaoxiong Wang, Mike Lambeta, Po-Wei Chou, and Roberto Calandra. Tacto: A fast, flexible, and open-source simulator for high-resolution vision-based tactile sensors. IEEE Robotics and Automation Letters, 7(2):3930-3937, 2022.
Tsun-Hsuan Wang, Pingchuan Ma, Andrew Everett Spielberg, Zhou Xian, Hao Zhang, Joshua B Tenenbaum, Daniela Rus, and Chuang Gan. Softzoo: A soft robot co-design benchmark for locomotion diverse environments, arXiv preprint arXiv:2303, 09555, 2023.
Zhou Xian, Bo Zhu, Zhenjia Xu, Hsiao- Yu Tung, Antonio Torralba, Katerina Fragkiadaki, and Chuang Gan. Fluidlab: A differentiable environment for benchmarking complex fluid manipulation. In The Fleventh International Conference Learning Representations, 2022.
11

---

Published as a conference paper at ICLR 2024
Jie Xu, Tao Chen, Lara Zlokapa, Michael Foshey, Wojciech Matusik, Shinjiro Sueda, and Pulkit Agrawal. An End-to-End Differentiable Framework for Contact-Aware Robot Design. In Proceedings of Robotics: Science and Systems, Virtual, July 2021. doi: 10.15607/RSS.2021. XVI1.008.
Jie Xu, Sangwoon Kim, Tao Chen, Alberto Rodriguez Garcia, Pulkit Agrawal, Wojeiech Matusik, and Shinjiro Sueda. Efficient tactile simulation with differentiability for robotic manipulation. In Conference on Robot Learning. pp. 1488-1498. PMLR, 2023.
Kuan-Ting Yu and Alberto Rodriguez. Realtime state estimation with tactile and visual sensing. application planar manipulation. In 2018 IEEE International Conference on Robotics and Automation (ICRA). pp. 77787777777 777877785,.0 2018. doi:
Wenzhen Yuan, Siyuan Dong, and Edward H Adelson. Gelsight: High-resolution robot tactile sensors for estimating geometry and force. Sensors, 17(12):2762, 2017.
Shaohong Zhong, Alessandro Albini, Oiwi Parker Jones, Perla Maiolino, and Ingmar Posner. Touching a nerf: Leveraging neural radiance fields for tactile sensory data generation. In Conference on Robot Learning, pp. 1618-1622 PMLR, 2023.
12

---

Published as a conference paper at ICLR 2024
A
APPENDIX
A.1 SIMULATION DETAILS
We implement our whole system with Taichi (Chen al., 2023) along with Python to benefit from its high computing performance and auto bitto-differentibbiity.. With Taichi, our system can switch between running with CPU or being accelerated by GPU by simply passing an argument initialize the Taichi environment. Taichi also supports automatic differential features for functions with explicit time integration. Therefore, considering the implementation difficulty and generalizability, our system is implemented with semi-explicit time integration, and without any extra effort, is fully differentiable and can be used for gradient-based trajectory optimization. The simulation pipeline for each simulation step can be seen in Fig. 6.
Compute internal elastic forces
Particles to grid
Collision detection
Compute external contact forces
Penalty-based contact model
Update tetrahedron node's 5 velocity
Grid operations
Time integration
Grid to particles
FEM-based sensor model
MPM-based object nadel
Figure 5: Simulation pipeline for each simulation step. Both the FEM sensor and MPM object have their pre-contact updates, and then we use a two-way coupling to handle collision and calculate contact forces. The contact forces are used in post-contact for both the FEM sensor and MPM object.
A.2 SYSTEM IDENTIFICATION DETAILS
Real-world data collection We collect sequences of contact data from the real world including the 6-axis force readings from robot arm end-effector, the poses Gelsight tactile sensor, and the corresponding tactile images from the tactile sensor. We set up the experiment by mounting a GelSight tactile sensor on the end-effector of an Ur5e robot arm and then controlling the robot arm to get the sensor in contact with tabletop surface. As discussed in (Yuan al., 2017), four general contact patterns are essential to capture and simulate for tactile sensors including contact under normal force, shear force, in-plane torque, and tilt torque. Therefore we collect three types of sequences of contact data: press-slide, press-twist-z, and press-twist-x. For each sequence, we start by pressing the sensor normally to a flat surface with a constant velocity of mm/s for 10 seconds to get in contact. Then we slide the sensor along the surface, twist it along the normal direction, or twist it along horizontal direction finish press-slide, press-rwist-z, and press-twel twist-x respectively with constant velocity of mi/s or 2 degrees/s for 10 seconds.
Gradient-based estimation We define the losses including the pixel-wised tactile marker distances the fuctile loss and three-axis force errors as the force loss between the simulated and ground truth data. Since the two losses are on different numerical scales, we aggregate them by scaling with weights 10:1 the final loss. We use Adam optimizer with 31 0.9, 82 0.999. Learning rate parameters are Ir = 20.0, (rkd = 20.0, lykt = 5.0, Ir = 5.0, Ir, = 50.0, Irx 50.0 depending their numerical scales. We run 100 optimization steps for each trajectory.
RNN-based estimation We use the Long Short- Memory (LSTM) model as the network architecture. The inputs of the LSTM module with the size of 2 136 use 136 tracked markers' 2D motions from tactile images, and three-dimensionae aggregated contact forces. We set the hidden layer size 256 and used the default settings for other parameters. We use
275. where
13

---

Published as a conference paper at ICLR 2024
linear layer after the LSTM module with input size of 256 and an output size of to predict the six parameters of the sensor material and the contact model. We generate simulated dataset that includes tactile marker readings, and three-axis contact force readings based on randomized system parameters. Our dataset has 2010 samples, 1800 for training, 200 for validation, and 10 for testing. Each sample's system parameters randomized within pre-determined ranges, which ensures their real-world applicability as shown in Table 6. The model was trained in batch size of 32 for 3000 epochs, and using an Adam optimizer with a learning rate of 0.001.
Parameter
Lower
Upper
Kn
10
100
100
400
50
150
Fc
5
20
800
1500
À
7000
10000
Table 6: Range of parameter randomization.
Random estimation We also provide random estimation randomized system parameters from our dataset and then compare the simulated tactile markers with the ground-truth markers either from the simulation the real world.
baseline. We use 10 sets of
Experimental details We evaluate the system identification with Limulation-to-simlaai (sim2sim) and real-to-simulation (real2sim). For sim2sim, create dataset by randomly sampling ten sets of parameters within appropriate ranges and simulating their corresponding tactile marker data. We validate different methods including baselines and ours by predicting parameters on this dataset. The Mean Squared Error (MSE) of the marker positions between the ground truth and the simulated ones with estimated parameters is used as the evaluation metric. We compute the mean and standard deviation of these ten average marker errors for the entire trajectory.
For real2sim, we collect the dataset from the real world and apply different methods including baselines and ours to estimate the parameters. Then we simulate tactile markers using the optimized parameters and evaluate the performance by computing the mean and standard deviation of the marker errors between the actual and simulated tactile marker data.
A.3 OPTICAL SIMULATION DETAILS
A.3.1 NETWORK ARCHITECTURE
40
Figure 6: Multi layer perceptron neural network architecture used for optical simulation inspired by (Mildenhall et al., 2021). We represent inputs with green blocks, hidden layers with blue blocks, and outputs with red blocks. Solid arrows represent ReLU activation, dotted arrows means no activation, and dashed arrows mean sigmoid activation. + represents vector concatenation.
A.3.2 TRAINING DETAILS
In our experiments optimize model using ADAM optimizer with fixed learning rate of 1e-5 for 500 epochs. Each batch consists of all the data from a single example image. Training takes approximately 45 minutes for 200 examples.
14

---

Published as a conference paper at ICLR 2024
Lr
ObjectRepose
CableStraighten
CascOpen
SurfaceFollow
GraspElastic
GraspPlastic
Irp
5c0
le 2
le3
5e -
5e 2
5e 2
Ira
1c3
le - 2
le1
5e -
le 5
le 5
Ir
N/A
le 2
N/A
N/A
2
5e 2
Table Learning rate of method in each task
ObjectRepose
CableStraighten
CaseOpen
SurfaceFollow
GraspElastic
GraspPlastic
lel
le
lel
le2
le2
5e
5e
12
le
5e 12
1c0
5c0
le1
Table 8: Coefficient of combined loss a and / of each task
A.4 DIFFTACTILE TASK AND EVALUATION DETAILS
A.4.1 TASK SETUP DETAILS
Reinforcement Learning (RL) We use object particles' state vector (t( and tactile sensor's pose (t) as state observations. Additionally, we use tactile markers' position 2D image my (w;; vz), three-axis contact force F() (Fz, Fy. F2), and contact location center ((L) by averaging all in-contact nodes' locations as tactile observations. Given that the total number of markers is 136, we downsample the number of object particles to four times of the number of markers, ensuring balanced dimensionality across different input segments. The input vector is formulated as either only state observation with additional tactile observation. Then it fed into a Multi-Layer Perceptron (MLP) policy network.
We use babel-b (Raffin et al., 2021)'s default PPO and SAC as our policy networks, Given an initial trajectory which is the same for all baseline methods, the policy network takes the input vector and outputs an action (t, of the sensor for each time step. Then we update the sensor's velocity v. (t)+ (t.(t). We constraint the actions in the range of 0-015,, 0. 0.15] for reasonable action size.
CMA-ES In each optimization step, we generate 20 new trajectories based the current trajectory with a standard deviation of 0.15 for a fair comparison with RL. We evaluate each new trajectory's loss and then update the policy based on the evaluation. This then informs the generation of the next optimization step's 20 trajectories. We used the same initial trajectory as RL and ran 100 optimization steps in total for each task.
Gradient-based Optimization (Ours) In each optimization step, we forward the simulation and calculate the defined loss, and then backpropagate the gradients from the loss the target optimization variables. We then update the target variables with Adam optimizer. To enhance optimization efficiency, use different learning rates for different optimization variables. The can be found in Table 7, where Irp is the learning rate for translation, Ir, is the learning rate for orientation, and Irw is the learning rate for the gripper's width. Note that for tasks where we use single tactile sensor, the value of (r,, is listed as N/A.
A.4.2 LOSS AND REWARD
During training, we assign lask-spec weights state and tactile losses, denoted as a and B. The final loss is then calculated as Ltotel and B are provided in Table 8. We use the losses discussed in Section 4.5 and Section 4.6 for optimization-baseee methods including algorithms, we subtract the cumulative loss of two consccutive steps to obtain each step's loss, and then calculate the reward to fit the settings of RL algorithms.
Latate + 3 Ltactite- The task-specific values for
gradient-bused method and CMA-ES; for model-free RL
A.4.3 METRICS DETAILS
We design task-specific metrics for evaluations. Metrics' mathematical formulas are shown in Table 10.
15

---

Published as a conference paper at ICLR 2024
Notion
Explanation
P(t)
3D position of the center of the object Aggregated three-axis force vector on the surface of the tactile sensor
F(t)
F.())
The shear force The normal force in the sensor coordinate frame The friction coefficient
sensor coordinate frame
(a(t)
p
((t)
The center location of the contact
the tactile sensor surface
((L)
The rotated angle of the object from its initial pose The signed distance field of the object The mass distribution of the object
SDP(1)
(()
(1ontace
The traveling distance of the sensor while being in contact with surface The number of object's particles
N
Table 9: Explanation of parameters used in loss and metric computation
Surface Follow
We evaluate the continuous in-contact distance deontocs(t) the sensor travels on the surface within a fixed timestep span. Here, longer distance means better results.
Cable Straighten Our metric the average displacement of each particle on the cable from its target horizontal position [p,(t) (tttrggtt)] A smaller value indicates more desired result.
Case Open We calculate the opened angle in degrees between the case lid and the horizontal tabletop (t).. The opened angle, due to gravity, potentially show negative value if the training results are suboptimal. Therefore, larger value suggests better performance.
Object Repose We measure the rotated angle in degrees of the object f(t) from its initial pose. A larger value this context indicates better performance.
Task
Mathematical Formula of the Metric
SurfaceFollow
deoonace (1)
>, (t (ttrgggt)]]
CableStraighthen
CascOpen
A(t)
ObjectRepose
8(1)
Table 10: Metries for four contact-rich manipulation tasks.
A.4.4 LOSS DETAILS
We design different losses used for different tasks to obtain state or tactile reward, shown in Table 11.
Grasping The losses used defined in Section 4.5. 7 and n) set to 0.1.
Surface Follow We use Lpos as the state loss and Lforee as the tactile loss.
Cable Straighten We use Leable the state reward which is the sum of the distance between the target position and the current position for each node on the cable. Tactile loss comprises L force + Tioe-
Case Open & Object Repose We use Langle as the state loss and forre as the tactile loss.
A.4.5 PARAMETER DETAILS
We apply optimized parameters including FEM-based sensor's Lamé parameters μ and in manipulation tasks since we only use one kind of tactile sensor. Other parameters vary depending on tasks since different tasks use different objects. Sensor-related parameters are shown Table 12. Object simulation parameters for different tasks shown in Table 13, where Satj is the object scale compared to the grid size in MLS-MPM, p is the density, Np is the number of particles one dimension of the space, / and À are Lamé parameters, and is the yield stress for the elastoplastic
16

---

Published as a conference paper at ICLR 2024
Loss
Equation
Lipoe
[|t(t) Plargee
Ldeform
"dact TyLmmsss
Ldiet
SDF(t) SDDF
Lmaas
[t Maargesl
Ledip
11&
Liforce
[()) Fearget ||
Lioe
[[((t) larget
Lungie
[(((1) Osarges|1
Leable
Σ; (p) pe(large()11
Table 11: Losses used for manipulation tasks.
Task
/FEMM/Pa
XFEMyPa
k. (Contact)
ka(Contact)
(CContact)
(CContact)
ObjectRepose
1.291e
9.201e
55.0
269.44
108.72
14.16
CableStraighthen
1.294e
9.201e
55.33
239.97
94.35
4.90
CaseOpen
1.294e
9.201e
34.53
269.44
108.72
14.16
SurfaceFollow
1.294e
9.201e
34.53
269.44
154.78
43.85
Grasping
1.291e
9.201e
55.33
239.97
94.35
4.90
Table 12: FEM-based sensor parameters and contact model parameters.
object. For the articulated object, we list the Lamé parameter for parts from the top to the bottom of the object.
A.4.6 TRAJECTORY OPTIMIZATION TRAINING Loss CURVES
We show the trajectory optimization training loss curves for four contact-rich manipulation tasks in Fig. For each task, we set 100 optimization iterations for fair comparison. From the curves, we show state tactile settings converge faster than the state-only settings which indicates the benefits of using tactile information on these manipulation tasks.
A.4.7 DISCUSSIONS ON CONTACT-RICH MANIPULATION TASKS
For RL algorithms, both PPO and SAC's losses did not decrease in 100 iterations. There are three potential reasons: 1. The amount data we used is insufficient for RL algorithms to learn within 100 iterations. 2. Our tasks include continuous contact, whereas RL algorithms operate discretized per-smallltime-step basis, making challenging to optimize. 3. RL algorithms general require more detailed reward designs while the current reward functions too simple to train proper policies.
For CMA-ES, find that training losses decreased slowly. This is because CMA-ES is sample- based optimization method and it is not as efficient as the gradient-based optimization method.
Task
Object Type
Sob
pl(g/cm²)
N₂
P/Po.
XPaa
alPa
ObjectRepose
Rigid
4.0
1.2
38
1.428e
5.714e
N/A
CaseOpen
Articulated
6.0
1.2
57
1.428c 1128e"/e//e
/et/c/
5.7146
N/A
SurfaceFollow
Rigid
8.0
32.0
76
1.428c
5.714e
N/A
Grasping Elastic
Elastic
2.0
1.2
19
3.428
5.714
N/
Grasping Elastoplastic
Elastoplastic
19
1.428
5.714
5
Table 13: based object simulation parameters
17

---

Published as a conference paper at ICLR 2024
Losscs-lteration Training Curve of Our Mothod
CaeaCpen bitde
Cattecrrnpphhen crats
CtyccPapeus olide
Icaess
10
86
00
044
011
34
24
0/2
0/
92
92
120
lardl
CaneeCpen SIMEE-Ha MD
Cidtacc vagetien (SMte-aadi))
2.4
02
02
02
leral
Figure 7: Training loss curves of trajectory optimization method for four contact-rich manipulation tasks. The top row shows the training with state-only observations and the bottom row shows the training with statc+tactile observations.
Frame per second (FPS)
Forward
Backward
FEM
Contact
MPM/PBD
ObjectRepose
13.11
8.93
91.38
199.85
18.38
CableStraighthen
21.63
12.10
35.01
267.41
419.14
CaseOpen
7.41
4.13
67.07
39.58
11.46
SurfaceFollow 25.03
18.75
30.88
297.08
592.01
Table 14: Computational runtime benchmark on four contact-rich manipulation tasks. Simulation (Forward), Gradient backpropagation (Backward), and each module's 's runtime during forward simulation are reported in averaged frame per second (FPS) over a trajectory.
does show the ability to optimize the trajectory but requires more than 100 iterations of optimization to converge.
Thus, we can conclude that our proposed gradient- gradient-based optimization method with differential physics has these merits: 1) Better data usage efficiency, 2) faster converge speed with the guidance of gradients, and 3) simpler loss function design. We additionally visualize the failure cases of RI. algorithms and CMA-ES on our project website.
A.5 COMPUTATIONAL RUNTIME OF THE SYSTEM
We report the averaged computational running speed of our system on four contact-rich manipulation tasks in Table. 14. From the table, we show the simulation speed (Forward) depends on different task settings and is significantly affected by the object simulation. The gradient backpropagation (Backward) speed is twice as slow as the simulation. This is because we optimize our system to be memory efficient. During the forward simulation, we save the states of each first simulation substep, and then during each backward step, we retract them and replay the corresponding forward step to fill in the rest substeps' states. FEM-based tactile sensor simulation can consistently run at high speed (greater than 30 FPS) even with two sensors in the CableStraighthen task. However, the simulation speed of objects varies such as the multi-material 4IPM-based object simulation is slower than others in the CaseOpen task, while the PBD-based cable simulation is the fastest. All experiments were conducted on a Ubuntu 18.04 with AMD Ryzen 7 5800x 8-core processor and Nvidia GeForce RTX 3060.
18

---

Published as a conference paper at ICLR 2024
A.6 REAL-WORLD EXPERIMENTS
A.6.1 EXPERIMENTAL SETUP DETAILS
We use a Gelsight tactile sensor for both system identification tasks and sim-to-real tasks. The sensor is manufactured in the laboratory with design flexibility. The soft elastomer made with SYLGARD 184 silicone elastomer, and in dome shape with an inner radius of 7.5 mm and an outer radius of 15.0 mm. The sensor uses an Arducam 180-degree fisheye camera to output tactile images.
A.6.2 EXPERIMENTAL RESULTS
To demonstrate our proposed simulator's fidelity, we conduct two sets of experiments in the real world. First, we demonstrate that the trajectories optimized with differential physics can deployed on real-world setups in Fig. 8. Here we show the sim-to-real transfers on SurfaceFollow and CaseOpen tasks. Then we further evaluate with a closed-loop grasp task by only using tactile sensing feedback. We train grasp stability prediction network in simulation by using sequence of tactile observations during grasping as inputs and predicting a binary output to indicate whether is a stable grasp slippage. The prediction is then used to guide the grasp adjustment. We directly use the trained model on a real-world deformable object grasp.
To train the grasp stability prediction network, we apply domain randomization and generate multiple trajectories in simulation with different parameter settings to improve the generalization of sim2real transfer. The process of trajectory generation is let the gripper close at the speed of Velase = 5 mm/s until the gripper begins to squeeze the object for Toontact frames, then we let the gripper lift the object for Tisse frames at the speed of Ulift = 1 mm/s. If the slipping distance between the object and the sensor is less than 0.75 mm, we label the trajectory as stable grasp. The parameters we used are shown Table 15, where Sokj and p are the scale and density of the object. Additionally, the object shape is chosen randomly from the object set of grasping experiments in Section 4.5. For the Lamé parameters of the tactile sensor, we use the results from system identification in Section 4.3 The Lamé parameters of the object are set μ 1.428e3. 5. The frequency of the system is 40 Hz.
We generate 50 trajectories of stable grasp and 50 trajectories of unstable grasp in total. We split the training/validaaten set in a ratio of 7:3. We use two LSTM layers and one MLP layer as the network architecture. We use the sequence of tactile markers' 2D motions as inputs for the model. Each frame's tactile markers' 2D motions are obtained by subtracting the initial marker positions from the marker positions in the current frame. Due the varying number of frames in different trajectories, we perform zero-pas pero-padding at the beginning of the trajectories, making the network input size (I., 136 2), where is the maximum number of frames among these trajectories. After training for 10 epochs, the success rate reaches 94.3% on the training set and 90.0% on the validation set.
We present our sim2real adaptive grasp policy for grasping a deformable object. Our goal is grasp the object with minimal deformation. We applied our trained model directly on the real-world setup to perform an adaptive grasp. We first attempt to grasp and lift the object with minimal force, feeding the sequence of tactile marker motions into our trained model. If the model predicts slippage, we tighten the gripper, or if the model predicts stable grasp, we continue lifting the object to the determined height. In Fig. 9, we show comparison of our approach with two baselines: 1) forceful grasp where the gripper tightly grips the object, and 2) light grasp where the gripper lifts the object upon contact. We can see that our method successfully grasps and lifts the deformable object with minimal deformation while the two baselines failed by damaging the object or causing slippage.
19

---

Published as a conference paper at ICLR 2024
Surface following
Case opening
Figure 8: Real-worke world experiments of surface following and case opening tasks. We deploy the optimized
trajectories from simulation directly to real-world setups.
Forceful grasp
Light grasp
Adaptive grasp from sim2real policy
Figure Real-world experiments of grasping deformable object. With hardcoded policics such forceful grasp or light grasp, they failed to grasp the deformable object with either damage or slippage. With our sim2real adaptive grasp policy, we can successfully grasp the object.
Parameter
Value
kn
[10,100]
ka
[100,400]
kt
[100, 250]
μ
[5, 80]
Sobj
[]
[0.1, 4]
T Tronsact
[0, 60]
Tisse
60
Table 15: Parameters for domain randomization on sim2real grasp policy training.
20


Reviews:
[{"review_id": "224BGWiBz4", "reviewer": "Reviewer_TYvh", "summary": "This paper presents a differential simulator for tactile sensors which work on the principle of light reflection from coloured deformable surfaces. The deformable surface is simulated by a finite element model, with contact forces based on penetration penalty. Surface normals of the deformed tactile surface are used by a neural network to predict the RGB colour of the reflected light. All these operations are differentiable\n\nThe paper presents experiments about system identification, grasping, and various manipulation tasks - all in simulation. The differential nature of the simulator allows gradient-based trajectory optimization for these tasks. Experiments show that this outperforms CMA-ES and RL.", "questions": "- What is the computational runtime of the proposed method, and how does it affect the intended applications?\n- Which simulation parameters are used for the manipulation experiments in Section 4.4? Are they the parameters identified from the real robot system?\n\n### After rebuttal\nI would like to thank the authors for addressing my concerns from the review during the rebuttal phase. I am raising my rating because of this.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Tactile sensors provide highly useful sensing streams for fine manipulation tasks. However, they have been difficult to simulate. This hampers the training of policies with tactile observations in simulation. The differentiability of the proposed simulation system can enable data-efficient system identification. Gradient-free methods can also be used for system identification, but they usually require more data, including real robot data synchronized to sim, which can be expensive.\n- Multiple experiments with multiple tasks show the general applicability of the proposed system.\n- The paper is well written and easy to understand.", "weaknesses": "- It is difficult to judge the accuracy of system identification based on the MSE in tactile markers location alone. Small errors can lead to large drops in downstream task performance. Therefore, system identification algorithms are usually evaluated by sim2real task performance [1, 2]. This paper lacks sim2real task performance experiments.\n- Is CMA-ES not applicable to the system identification task (Section 4.1)? If it is, please discuss why it was not used as a baseline.\n- A lack of discussion of computation time, especially the FEM-based deformation module.\n- A lack of implementation details like RNN structure for parameter identification, optical prediction network architecture (Section 4.2), mathematical formulations of the reward functions used for the manipulation tasks (Section 4.4). \n\n### References\n1. \"Generation of GelSight Tactile Images for Sim2Real Learning\" RA-L\n2. \"Efficient tactile simulation with differentiability for robotic manipulation\" - PMLR", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SOJ8xCyxXS", "reviewer": "Reviewer_Wssn", "summary": "This paper is about creating a differentiable tactile stimulator that supports contact rich tasks. Having such a simulator is important to learn robust policies using tactile. The simulator consists of 4 components: 1) sensor deformation through FEM 2) Optical simulation model that maps sensor deformation to rgb reflected color. 3) Objects are modeled using least square material point and position based dynamics. 4) penalty based contact model which goes from deformation, represented by SDF, to forces. They evaluated the simulator in 3 tasks: system identification which basically tries to estimate the sensor params from a set of collected real sensor data. The second task is grasping fragile objects and finally manipulating non-rigid objects such as straightening a cable.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Having differentiable simulator for contact-rich tasks unlock a lot of new capabilities. \n- The paper covers experiments in wide range of applications.", "weaknesses": "I would have liked to see application of the method for a contact-rich manipulation task in the real world.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EHizAFUwKx", "reviewer": "Reviewer_EisY", "summary": "The paper presents DiffTactile, a simulator that uses FEM to simulate soft tactile sensors such as Gelsight, supports elastic, rigid, elasto-plastic, and cable-like objects, and is differentiable. The authors demonstrate the use of the simulator for three tasks. First, in system identification, the goal is to use real-world tactile observations to optimize the simulator’s physical parameters, and then compare the tactile marker and force reading to that of real data. In optical simulation, the visual quality of the optical simulation is examined. In grasping, the task is to use a parallel jaw gripper equipped with tactile sensors to see if tactile feedback is helpful. Finally in manipulation, four tasks are performed: surface following, cable straightening, case opening, and object reposing.", "questions": "Please see my points in \"weaknesses\". In addition: \n- What differentiates the “grasping” task from the “manipulation” tasks? I think that the manipulation tasks are nice demonstrations of the types of tasks that can be simulated using DiffTactile, but I don’t quite understand what the “grasping” task is trying to show that is not being illustrated by the “manipulation” tasks, as it seems to me that it could easily become the fifth “manipulation” task.\n- Can you provide some intuition or qualitative visualization for why the baselines like PPO and SAC don’t perform well on the manipulation tasks? What are the failure modes?\n- (nit): I recommend adding periods after paragraph section headings or otherwise distinguishing them from the following text.\n- (nit): For the “experimental results” on page 9: “For case opening and object reposing, we define the metric as the opened angle of the lid and the orientation of the object.” I think it’s not quite accurate to refer to the orientation of the object as a metric.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Developing accurate simulation for tactile sensors is quite challenging, particularly for optical tactile sensors. However, the demand is quite large, as it can be expensive or challenging to work with real tactile sensing hardware. This simulator has many features including simulation of many object types and optical simulation, which are not present in existing works and may make it valuable for the development of robotic tactile sensor applications.\n- The optical simulation results look quite impressive and match well with the real readings. \n- The introduction of the manipulation tasks is a nice demo of the types of tasks that can be modeled and learned in this simulator.", "weaknesses": "- In the system identification task, it seems like the pixel-wise tactile marker mean squared error is extremely high for the real to sim setting, or at least, the standard deviation or standard error (which one is it? It’s not labeled) is much larger than the differences between the different methods, including the random method. This is rather concerning, as it seems to indicate that the system identification method is not very effective at reducing the sim-to-real gap.\n- Rather than a comparison between using or not using tactile sensing for grasping, which has been validated in prior works, or comparison between different methods for optimizing policies to solve the manipulation tasks, I wish the paper focused more on evaluating how realistic and accurate the tactile sensing simulation is, as well as sim to real applications. I think those are the things that will really impact whether or not practitioners can rely on this simulator to generate conclusions that will hold in the real world.\n- The organization of the paper is slightly confusing: I think it would be easier to understand if the tasks were introduced closer in the text to where the results are presented. In general I think the clarity of the writing could be improved, for example, to be more explicit about when tactile signals are real or simulated (for example, in Section 4.2, is the training data real?)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zAgdF8zXn1", "reviewer": "Reviewer_Zvck", "summary": "The paper presents a new tactile simulation framework for soft optical sensors applied on robotics manipulation tasks. This fully-differentiable pipeline is then used for system identification of sensor properties, and gradient-based trajectory optimization. This framework is the first differentiable simulation that supports both soft optical sensors and soft object interaction.", "questions": "1. Were the constraints applied to the reinforcement learning methods for trajectory planning also applied to the gradient-based optimization? \n2. In Table 1, should the Tacchi method not also be differentiable since they are implemented in Taichi?\n3. How efficient is it to simulate rigid objects as elastic using MPM and then applying rigidity constraints? Are there any plans on extending the simulator to use rigid-body or articulated-body solvers?\n4. Are the results found from the SysID parameters used for the follow-up tasks?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The authors positioned the framework well in context to the state of the art, and it is clear what improvements the new simulation offers. The experiments show why the differentiability is beneficial and how the optical simulation compares to previous work and on real-world experiments. A wide range of tasks were tested to verify the results for the optical sensors, and show a significant advantage of using differentiable tactile simulations.", "weaknesses": "1. While the accuracy of the optical sensor model is verified, the simulation itself would benefit from having a comparison to real-world experiments as well. Questions such as how well the dynamic behavior of soft objects match reality (bouncing objects), or how well the contact model applies to objects sliding/being pushed under friction. These would be more of a benchmark on the FEM-MPM-PBD simulation, but this could nevertheless broaden the applicability of the framework.\n2. How does the simulation compare when simulating articulated bodies? For example the lid-opening task, is the hinge a soft body or a joint? How well does this compare to modeling real-world objects?\n3. Adding some runtime reports, at least in the appendix, would be appreciated from a practitioner's perspective, since the choice for using FEM only for the sensor likely stems from high accuracy but high computational complexity as well. Hence MPM or PBD was used to simplify object simulation, is this correct? If so, it would be interesting to see how expensive each part of the simulation is, where simplifications are necessary when used in practice. Were any of the learned grasping or manipulation policies applied on the real robot?\n4. Continuing on the topic of runtimes, for the trajectory optimization tasks in manipulation, it would be interesting to see how many iterations/computational resources each method was given to converge, was it the same for each, or was each method run until convergence?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)