Efficient Tactile Simulation with Differentiability for Robotic Manipulation
Authors: Jie Xu
Sangwoon Kim
Tao Chen
Alberto Rodriguez Garcia
Pulkit Agrawal
Wojciech Matusik
Shinjiro Sueda
Venue: CoRL 2022 Poster (2022)
Keywords: Tactile Simulation
Tactile Manipulation
Differentiable Simulation
Sim-to-Real

Abstract: Efficient simulation of tactile sensors can unlock new opportunities for learning tactile-based manipulation policies in simulation and then transferring the learned policy to real systems, but fast and reliable simulators for dense tactile normal and shear force fields are still under-explored. We present a novel approach for efficiently simulating both the normal and shear tactile force field covering the entire contact surface with an arbitrary tactile sensor spatial layout. Our simulator also provides analytical gradients of the tactile forces to accelerate policy learning. We conduct extensive simulation experiments to showcase our approach and demonstrate successful zero-shot sim-to-real transfer for a high-precision peg-insertion task with high-resolution vision-based GelSlim tactile sensors.

Efficient Tactile Simulation with Differentiability for Robotic Manipulation
Jie Sangwoon Kim', Tao Chen', Alberto Rodriguez', Pulkit Agrawal', Wojeiech Matusik', and Shinjiro Sueda?
Massachusetts Institute of Technology TTexs A&M University
Efficient simulation of tactile sensors can unlock new opportunities for learning tactile-based manipulation policies in simulation and then transfer- ring the learned policy to real systems, but fast and reliable simulators for dense tactile normal and shear force fields are still under-explored. We present a novel approach for efficiently simulating both the normal and shear tactile force field covering the entire contact surface with an arbitrary tactile sensor spatial layout. Our simulator also provides analytical gradients of the tactile forces to acceler- ate policy learning. We conduct extensive simulation experiments to showcase our approach and demonstrate successful zero-shot sim-to-real transfer for high- precision peg-inserteee peg-insertion task with high-resolution vision- GelSlim tactile sen- sors. The videos and code are available at: http:/tacciilesim.cos wwt..maca.iimi.ii.........
Abstract:
Keywords: Tactile Simulation, Tactile Manipulation, Differentiable Simulation, Sim-to-Real
1 Introduction
Just as humans heavily rely on rich and precise tactile cues for dexterous grasping and in-hand manipulation tasks, robots can also utilize tactile cues as an important source of sensing for inter- acting with the surrounding environments, especially when the visual information is unavailable or occluded. With the recent development of various tactile sensors capable of generating dense normal or shear load information [1, 2, 3, 4), researchers have been exploring how to leverage this important mode of information for robotic manipulation tasks. With the dense tactile normal load field, the static spatial relation between the object and the robot manipulators can easily be inferred, which is useful for tasks such as edge following [5], pose estimation [6]. object reconstruction and recognition [7, 8]. On the other hand, the dense tactile shear force feedback more readily gives rich information about the dynamic tangential motions between the object and the manipulators, and thus can be utilized in tasks such as stable grasp [9], precise insertion [10, 11], and slip de- tection [12. 13, 14, 15]. However. most of the tactile manipulation work still requires significant amount of human effort on real hardware systems for collecting data, cleverly building automatic resetting mechanisms, and carefully designing the learning strategy [10]. Such manual work can be time-consuning, cost expensive, and more importantly unsafe during policy exploration.
Due to its capability to replicate the real world with high fidelity and low cost, physics-based simu- lation has become powerful recipe for learning robotic control policies [16, 17, 18, 19]. Previous work has demonstrated that the policy can be efficiently learned in simulation and successfully trans- ferred real robots via proper sim-to-real techniques [20. 21, 22, 23]. Despite the prevalence of simulation and the importance of tactile sensory in robotics, physics- bhysics-based simulation is still under- explored to efficiently simulate dense tactile normal and shear force fields for robotic applications. Most popular simulators [18, 19] only support force-torque sensors which are attached to each robot link. only producing the contact force values at few points on each body. Although one can acquire a dense tactile force field by attaching many small cuboids to the robot body and querying the force sensor on each small cuboid from simulation [24, 25]. the obtained tactile force values are usually sparse and are unable to match the uniform force distribution on real elastic tactile sensor such as GelSlim [1]. While researchers have also tried simulating realistic tactile feedback via pure ge-
6th Conference on Robot Learning (CoRI. 2022), Auckland, New Zealand.

---

ometric methods [5. 26]. such methods typically only compute the normal tactile force and cannot simulate the tactile effects in shear directions. On the other hand. the tactile shear forces have been successfully simulated via the finite element method (FEM) [27, 28, 29] or data-driven approach [30], but these simulators suffer from expensive computation costs, and cannot be easily used for data-hungry policy-learning approaches such as reinforcement learning (RL).
We present novel tactile simulator that can efficiently and reliably simulate both normal and shear lactile force fields covering the entire contact surface. We build upon rigid body dynamics formula- tion and develop a fast penalty- batee---bdeddde tactile model which can run at 1000 frames/s for ball-rolling experiment with high-resolution tactile field on a single core of Intel i7-9700 CPU. Our tactile model can reasonably approximate the soft contact nature of soft tactile sensor material such as the elastomer used in GelSlim [31], generate dense tactile force fields (e.g., the dense marker array on GelSlim), and compatible with arbitrary tactile sensor spatial layout (i.e., flat plane, hemisphere, etc.). Furthermore, our compact tactile formulation is differentiable, which allows the simulator to provide fast analytical gradients for the entire dynamics chain. We conduct extensive experi- ments in simulation to demonstrate the capabilities of our tactile simulator, including policy learning with reinforcement learning algorithms and gradient-based algorithms. We also conduct a zero-shot sim-to-real experiment for a high-precision tactile-based peg-insertion task, demonstrating that our simulator provides realistic tactile simulation.
2 Related Work
While there have been many physics-based simulators developed to simulate various types of robots, efficiently and reliably simulating dense tactile sensing fields is less explored. As mentioned above, most robotics simulators such as MuJoCo [18] and PyBullet [19] only support force-torque sen- sors that are attached to each robot link. While it is possible augment these simulators with high-resolution tactile forces, they become computationally cumbersome. In order to acquire more realistic and dense tactile forces, Narang et al. (27, 28] and Ding et al. [29] use soft materials to model the tactile sensors and apply FEM to simulate the deformation and force fields of the tactile sensors. Despite the high fidelity of the simulated tactile feedback, these simulators suffer from expensive computation cost and are primarily used to collect supervised tactile dataset instead of learning policies which are typically data-hungry and requires fast simulations. Dision-based tactile sensors produce high-resolution tactile feedback. To simulate vision-based sensors, Wang et al. [26] and Church et al. [5] use PyBullet [19] and render the intersecting part between the object and the tactile manipulator
depth images, from which tactile information is generated. However, such purely geometry- based approaches cannot simulate the tactile effects in the shear directions such as the marker displacements of GelSlim. Si and Yuan [30] compute the marker displacement field by presenting a superposition method to approximate the FEM dynamics. While they are able to simulate the tactile shear effects, the speed of the simulation is still slow, and control tasks demonstrated. Bi et al. [32] build an efficient simulation specialized for a tactile-based pole swing-up task with a customized vision-based tactile sensor, but the proposed technique not readily exten- sible to simulate other types of tasks and tactile sensor types. Similarly to our work, Habib et al. [33] use a paring- model to simulate tactile normal forces, and Moisio al. [34] use a soft bristle deflection model for simulating the tactile forces. However, there are no control tasks demonstrated to be learned with the presented simulators and no gradient information is available. In contrast to these previous works, we present a generic simulator with analytical gradients for tactile forces by leveraging the penalty-based rigid body dynamics, and we demonstrate that our simulation is efficient enough for policy learning, and simulated tactile force field can be successfully used for a sim-to-real task on the high-resolution vision-bant based GelSlim sensor.
3 Method
We now present our approach to simulate tactile forces for real-world tactile sensors. In $3.1, 1, we introduce our flexible representation for tactile sensors. In $3.2-3.3, we present our penalty-based tactile model for simulation and derive the analytical gradients of the dynamics. In $3.4, we describe our intermediate tactile signal representation for the sim-to-real transfer of the policies.
3.1 Tactile Sensor Representation
2

---

tactile
Each tactile point on sensor pad is represented by tuple (B,, &,) as shown in Fig. B, is the rigid body the tactile point attached to, and E, E SE(3) is the position/orientation of the point in the local coordinate frame of the body, with the æ;; and yi axes in the shear-direction plane and the axis along the normal tactile direction. (These axes are the same for all points for a planar sensor pad.) Finally, & are the simulation parameters of the penalty- -based tactile model, which will be introduced later in $3.2. Our representation of tactile points is flexible, allowing us to specify any number of points in arbitrary geometry layouts robot, and each tactile sensor can have its individual configuration parameters.
point i
E
Figure 1: Tactile Sensor Representation.
3.2 Penalty-based Tactile Model
We use penalty-based tactile model to characterize the force on each tactile point. For each point (B,, &,). we use the following contact model [35] obtain the contact force the tactile point's location represented the local coordinate frame B,. (For brevity, we drop the subscript i.)
fn (-k,, + kgd)dn.
f: ||elll
min(ke]/v,,l) pllff.l))
(1)
where fn. is the contact force at the tactile point along the contact normal direction n. and f. is the contact friction force the plane tangential to the contact normal direction. The scalar (nonposi- tive) is the penetration depth between the point and the collision object, and d its time derivative. The vector Ut is the relative velocity at the contact point along the contact tangential direction. Scalars kn, ka; & i are contact stiffness, contact damping coefficient, friction stiffness, and coeffi- cient of friction respectively, and they together form the simulation parameters of the tactile point: L.e, for the jth tactile point, & = (ki kig ki, μ² j. After the frictional contact force is computed for each point f fn + ft, transform this force into the local coordinate frame of the tactile point to acquire the desired shear and normal tactile force magnitudes:
Tex = f x. where æ, y, 2 the axes of frame E.
Tey = fTy. y. Ta =
(2)
Our penalty-based tactile model can be integrated into any simulator as long as the required values, such the world-frame location of the tactile points, the contact normal, the collision penetration depth and its time derivatives, can acquired from the simulator. We implement our tactile model in C++ and integrate it into differentiable RedMax (DiffRedMax) [35, 36) since DiffRedMax is open- source and readily provides all the required information for our computation, and more importantly, its differentiability allows to make our tactile simulation differentiable with moderate amount of modifications its backward gradients computation.
3.3 Differentiable Tactile Simulation
Since we use an implicit time integration scheme for forward dynamics, the core step of gradient computation is to differentiate through the nonlinear equations of motion. We start by formulating fiiie-horizznn tactile-basee policy optimization problem:
H
minimize 0
L = Σ
Culi (ui, qr ve(q:))
(3a)
1=1
s.t. g(q:-1, ue qt) =
(Equations of Motion)
(3b)
u, = To (Q: V:-1
(Policy Execution)
(3c)
Here, H is the task horizon, Le is step-wise task-dependent reward function, is the action (e.g., joint torque), q is the simulation state (i.e., joint angles), and v is the derived auxiliary sim- ulation variables (e.g.. fingertip positions) which themselves are function of q. Eq. 3b describes the nonlinear equations motion ($A.1). Eq. 3c represents the inference of the control policy Te to obtain the desired action given the partial observation of the simulation state q, partial observation of the simulation computed variables v, and the tactile force values from Eq. 2.
We compute the gradients dC//d0 = (aL/au (0u,/00))))) for policy optimization. We embed our simulator as differentiable layer into the PyTorch computation graph and use reverse mode differ- entiation to backward differentiate through dynamics time integration. The first gradient, DC/Ou,,
3

---

Tarile Foeres Nantliiod Floe Maps
Policy
Simulation (Triiinng)
Pose
Adjustment
Real Robot (Testing)
Images
Digdacnnerts
Figure 2: Sim-to-Real Pipeline for Insertion Task ($4.5), Gray Box: During training. we convert the tactile force output from the simulator into the normalized flow map representation (shaded in green). Yellow Box: When executing the policy real system, convert the sensor output into the same normalized flow map. This intermediate representation then treated the observation input neural network policy to output the pose adjustment for the next attempt. Here we only visualize the tactile output from one tactile sensor pad.
which includes the simulation dynamics and tactile derivatives, is derived analytically, shown below. The second gradient, 0u,/ 00, computed by PyTorch's s auto auto-differenniitiin.
We compute the gradients in reverse order. At each time step t. we derive ac/Ou, given the analyt- ically computed gradients with respect to the system states (0L/0q.) auxiliary variables (OL/Ov,), (v;,, and tactile forces (0L/0T,) (we drop the terms related to the time integrator here for brevity; see the full derivation $A.2-A.3)):
ac Du:
ac,
OC dv:
OC aT.
OT, dq:.
dqu
+ OT, dq.
du,
0q.
ov. 0q:
aq: 0q.
Ou,
(4)
-A-'D
The right-most derivative can be computed by applying the implicit function theorem on Eq. 3b, which gives us 0q:/ôu, = - -(0g/0g ((0g//4)--1agg (ag/ Writing this as 0q:/Ou, -A-1 and combining with Eq. 4, we first solve the linear system A = b for and then we compute the final gradient as 0C/Ou: = a c D using the adjoint approach (more details in $A.3).
3.4 Normalized Tactile Flow Map for Sim-to-Real
We use the GelSlim 3.0 sensor [4], which utilizes small markers track motions in the shear di- rection, to demonstrate the sim-to-real capability (84.5). There is an unavoidable sim-to-real gap between our simulator that emulates tactile forces and the physical GelSlim sensor that relies on imaging. In this section, we demonstrate how we overcome this gap by constructing common intermediate tactile representation for policy input observation. We assume that the stiffness of the sensor along different shear directions is isotropic and that there exists a linear relationship between the displacement and the contact shear forces (Tex and Tey from Eq, 2) at each tactile point. We connect these two different sensor output formats via unitless normalized tactile flow map repre- sentation.
Specifically, we use the raw tactile sensor images from the past k steps from the n tactile sensor pads on real robot as our policy observation 7. image I. As shown in Fig. 2, we first detect and identify the marker positions in each image, and obtain the marker displacement field TTlkk, E xxxx displacement
by subtracting the marker positions in the rest configuration from their positions in the deformed configuration, where r and are the rows and columns of the tactile marker array on each sensor pad, with each marker giving us the and displacement information. Then normalize the displacement field so that the maximal length of the marker displacement across all tactile points and images is of unit length; iee.
T [1kk,1mm displacement
ml1kk11m1)
(5)
normalized
max (max r.c (I/T displacemem (r, e))l),
where e ensures that the output is zero when there is no any displacement on the markers (i.e., no contact). We assemble these flow maps into single tensor T Tnomaaliedd E IR #xnxrxex2 which is

---

I. Ball Rolling Experiment
II. Stahle Grasp Task
III. D'Claw Cap Opening Task
(a)
(c)
Figure 3: (I) Ball rolling experiment: The tactile sensors are installed on the lower surface the pad. The depth map of the tactile normal forces is shown in (b). The tactile force field is shown in (c) with the arrow denoting the shear forces and the color denoting the magnitude of the normal force. (II) Stable Grasp Task: The bar is composed of 11 blocks with random densities (the deeper the color, the heavier the block). (a) An unsuccessful grasp results rotational patterns the tactile force field and (b) successful grasp requires gripper to adjust the grasp location to center mass the bar. (III) D'Claw Cap Opening Task: The tactile sensors (white dots) are installed at the three hemisphere fingertips of the hand. We map each tactile point at one fingertip onto 2D image plane and visualize the tactile forces field of three fingertips the right.
normalized tactile flow map representation. For the tactile shear forces {T& [1nk, T.(:2,11))) acquired from the simulation, we conduct the same normalization process as in Eq. 5.
Intuitively speaking, the normalized tactile flow map provides directional information about the relative motion of the markers induced by the contact forces, and also keeps the relative tactile load magnitude relationships among different sensors and different time steps so as to preserve meaningful spatial and temporal information about the contact. For our sim-to-real experiments, we only use the shear directional information from the sensor, but the same technique can also be applied to the normal directional information by normalizing the depth map of the contact surface reconstructed from the GelSlim image [4] across different frames and different sensor pads.
4 Experiments
We conduct extensive tactile-based experiments demonstrate the capability of our approach." We investigate the following questions: ($4.1, $4.2) Can our simulator reliably simulate the high- resolution tactile force field at high speed for RL algorithms? ($4.3) Does the differentiability of our simulator provide advantages in policy learning? (64.4) Is our tactile sensor representation flexible enough for sensors with arbitrary geometrical layouts? ($4.5) How does our simulated tactile force field compare to the tactile feedback from real sensors, and does our normalized tactile flow map representation help to transfer the policies learned in the simulator real robot?
4.1 Speed and Reliability: High-Resolutie Tactile Ball Rolling Experiment
We design ball rolling experiment to show the efficacy of the tactile force field generated by our simulator and to test the simulation speed. The simulation setup is shown in Fig. 3(I). A high- resolution tactile pad (200 200 markers) touches the marble ball and moves around. The simu- lation step size h = 5 ms, and we compute the tactile force field every 5 steps (i.e., 40 Hz). Fig. 3(I) also shows the normal tactile force (represented by a depth map) and the tactile shear forces ac- quired from our simulator. For this example, our simulation runs at 1050 frames per second (FPS) on single core of Intel Core i7-9700K CPU. The simulation speed can further accelerated by simply parallelizing it across multiple CPU cores, as we do in the RI. experiments.
4.2 RL Training: Tactile-Based Stable Grasp Task
Our tactile simulator provides the shear force information on the contact surfaces, which is critical for many manipulation tasks. Inspired by the setup in [9], we show the usage of shear force infor- mation for control and the effectiveness of our tactile simulator in parallel-jaw bar grasping task.
See also the supplementary video.
5

---

As shown in Figure 3(II), the task requires a WSG-50 parallel-jaw gripper to stably grasp bar with unknown mass distribution in fewer than 10 attempts. The gripper has two tactile sensors with a tactile marker resolution of 13 x 10. The bar is composed of 11 blocks where the density of each block randomized. The total mass of the bar is in the range [51, 120] g- We consider a grasp to be a failure if the bar tilts more than 0.02 rad after the gripper grasps bar.
We use RL train control policy that determines the grasp location. The initial grasp location is the geometric center of the bar. Based on the tactile sensor readings (the only observation input to the policy), the policy outputs a delta change the grasping location. The policy is a shallow CoNv-RLUU-FC-
CNN Mony-ReLU-Max/ee
that takes as input the two tactile sensor readings. We train the policies with PPO [37] using 32 parallel environments with 20K environment steps in total. We train the policies with 3 different random seeds and test them 320 times. The success rate 98.5 + 1.8%. The average number of attempts taken to stably grasp the bars is 2.1, meaning the policy can compute the correct grasping location after single failed attempt in most cases.
4.3 Differentiahility: Tactile-Based Box Pushing Task
Pos. ERROR
ORI.
ERROR
250
Drivileeee
0.037 1 0.1022m
0.043
10003
Goal
GD- GD-Redhced
0.126 + 0.009
0.235 = 0.0121
Bandom
Reward
PPO- Tachile
0.123 + 0.034
0.241
-.1123
-3000
GD- Tactile
0,058 : 0.003m
0.074 à 0.020
GD-Priillgee
GD Reduced
Pactic
GD-Tactile
Table 1: Metrics comparison on box pushing task. We compute the final po- sition/orientationn errors of the best policy in cach run and average the metrics from five runs for each policy variation. GD- Privileged gives a reference of the best pos- sible metrics, and without the privileged state information of the box, our GD- -D-Tacllle achieves much better position error and ro- tation error than the other two variations.
Steps (x11f)
(1)
(b)
Figure 4: Tactile-based box pushing task. (a) The goal of the gripper policy is to use its tactile feedback push a box to a randomized target position/orieetaaionn. A time-varying external force is randomly applied to the box during the task. (b) the training curve for each policy variation is averaged from the five independent runs with different random seeds.
In this experiment, we design a box pushing task similar to [5] to demonstrate how we leverage the provided analytical gradients to help learn factile-bsed control policies better and faster.
Task Specification As shown in Fig. 4(a), the task here is to use the same WSG-50 parallel jaw gripper as $4.2 (with only one finger kept) to push the box to a randomly sampled goal location and orientation. The initial position of the box is randomly disturbed. A random external force is applied continually on the box, which changes every 0.25 s. More details of the task are in BB.3.
Comparing Policy Learning Algorithms We train the control policies through four different combinations of learning algorithms and observation spaces. GD-Privileged: This variation uses the gradient-basse optimizer Adam by utilizing the analytical policy gradients computed from our differentiable simulation. The policy observation contains all the privileged state information of the gripper, the box, and the goal. This policy provides an upper-bound performance reference. GD-Reduced: Similar to GD-Priiilegd,, except that the observation space only contains the state information that can be acquired on real system such as the gripper state and the goal. GD-Tactile: Other than the state information used in GD- Redducd we also include the tactile sensor readings in the policy input. The policy is trained using the analytical policy gradients. PPO-TactiAe Similar to GD-Tactile, but the policy is trained by PPO.
All the policies are trained to maximize the same reward function ($B.3). We run each variation five times with different random seeds, and plot their training curves averaged from five runs in Fig. 4(b). We also randomly sample 300 goal poses and measure the final position and orientation errors between the box and the goal pose of the best policy from each run, and report the average metrics across five seeds in Table 1. The results show that when neither state information of the box nor tactile information is available (i.e., Reddudde,,,, the policy cannot reliably push the box to the target location since the gripper has no clue when the box goes outside of the control of the gripper

---

due to the random initial box position perturbation and the random external forces. With tactile information feedback (i.e., PPO-Tacciie and GD-Tactile), the gripper has tactile information to keep the gripper touching the box, allowing it to push the box to the goal effectively. However, the high dimensional tactile observation space results in higher computational costs with PPO, which relies on stochastic samples to estimate the policy gradients. In contrast, with the help of our differentiable tactile simulation, GD-Tactile makes use of the analytical policy gradients and leads to faster policy learning and better policy performance.
4.4 Flexibility: Tactile Sensor Simulation on Curved Surfaces
To demonstrate that our method supports tactile sensors on curved surfaces, we train D'Claw [38] tri-finger hand to open a cap on a bottle. We put the tactile sensors on the three rounded fingertips as shown in Fig. 3(III). The sensor layout on each fingertip is hemisphere, and we use 302 evenly- spaced tactile markers. We build coordinate mapping to project the marker positions on the 3D surface into 20 x 20 2D array (with some empty values around the boundary). Fig. 3(III) shows that our simulation can produce reliable and realistic tactile sensor readings on a rounded fingertip.
The task is to open a cap using the D' O'Claw hand. The position and the radius of the cap are random- ized and unknown. There also unknown random damping between the cap and the bottle. The task is considered success if the cap is rotated by 45 The only observation data that the policy gets are the angles of each joint, fingertip positions, and tactile sensor readings. This task similar to how we open caps by just using proprioception sensory data and tactile feedback on the fingers without knowing the exact size and location of the cap. The policy outputs the delta change on the joint angles. We again use PPO to train the policy (a shallow CNN) using 32 parallel environments. To show that the tactile sensors are useful in this task, we also train a baseline policy (a simple MLP policy) where the policy only takes input the joint angles and fingertip positions. With tactile sensor readings, the policies learn significantly faster and achieve an 87.3% success rate, while the policies only achieve a 59.7% success rate when tactile sensor information is unavailable. More details about task setup and results are provided in $B.4.
4.5 Zero-Shot Sim-to-Real: Tactile RL Insertion Task
In this experiment, we show the quality of the simulated tactile feedback when compared to the tactile sensing obtained from the real system, and demonstrate how to do zero-shot transfer for the policies learned in simulation to the real robot via our normalized tactile flow map representation.
Task Specification We experiment on the tactile-RL insertion task similar to [10]. In this task, a gripper (same as in $4.2) is controlled to insert a cuboid object into a rectangle-shaped hole with random initial pose misalignment. The insertion process is modeled as an episodic policy that iterates between open-loop insertion attempts followed by insertion pose adjustments (shown in Fig. 2). The robot has up 15 pose correction attempts, and the robot only has access to tactile feedback from the sensors installed on both gripper fingers. For the real robot system, we use a 6-DoF ABB IRB 120 robot arm with WSG-50 parallel jaw gripper. On each side of the gripper finger, we mount a GelSlim 3.0 tactile sensor that captures the tactile interaction between the fingers and the grasped object high-resolution tactile image. More details are in $B.5.
This task is more challenging than the stable grasp task ($4.2) because it not only needs to recognize the rotational pattern of the tactile field when the object contacts the front/back edges of the hole, but also needs to leverage the different magnitude relationship of two sensors' outputs to tell whether the object hits the left or the right hole edge (Fig. 5). It becomes even more challenging when the object hits the hole at four corners, because the robot must recognize nuances in the tactile pattern to decide the insertion pose adjustment. Therefore, this task requires high-quality simulated tactile force field in order to transfer the learned policy to the real system successfully.
Policy Learning via RL We train the control policies with PPO [37] for three types of misalign- ments. Rotation: The object initialized at the hole's center and has random rotation misalignment around the vertical axis. The action of the policy is the angle adjustment in the next insertion at- tempt. Translation;
The object has randomly initialized translation misalignment to the hole and no rotation misalignment. The action space in this case is two dimensional for the translational correc- tion on the z-y plane. Rotation & Translation: The object has both rotation and translation initial misalignments. The action space of the policy is three dimensional.
7

---

(a) contact the left edge
(b) contact the right edge
(c) contact the back edge
(d) contact the front edge
Figure 5: Comparison of the normalized tactile flow maps. The flow maps in the top blue boxes are from simulation (with noise added), while the flow maps in the bottom red boxes are produced from the real GelSlim sensor, each box, the two flow maps (left and right) are for the two tactile pads on the two gripper fingers.
During training, convert the simulated tactile force field into normalized tactile flow map rep- resentation ($3.4), and treat the resulting tactile flow map as 13 x 10 flow "image" with channels (2 sensors and 2 shear components of tactile forces). We model the policy by convolutional RNN to leverage more information from previous attempts. For better sim-to-real performance, we also apply the domain randomization technique [39] on contact parameters, tactile sensor parameters, grasp forces, grasp height and tactile readings, to increase the robustness of the learned policies. More details of policy learning are provided in $B.5.
Experiment Results We first qualitatively compare the normal- ized tactile flow maps generated by simulation and by real GelSlim sensors. We plot the normalized tactile flow maps at four represen- tative contact configurations (i.e., object contacts different edges of the hole) [10] in Fig. 5, which shows that our simulation able to produce highly realistic patterns in those contact configurations. We then deploy the policies learned in simulation on real hardware and quantitatively test its zero-shot performance by conducting 100 in- sertion experiments under different initial pose misalignments. As reported in Table our zero-shot policy transfer achieves 100%
TASK
SUCCESS
ATTEMPTS
R T R&T
100%
1.53
100%
2.33
83%
4.81
Table 2: Zero-shot sim-to-real performance of the tactile RL insertion policies.
success rates on Rotation and Translation tasks. We also calculate the average number of pose cor- rections for successful experiments. The average number of pose corrections is 1.53 for the Rotation task and 2.33 for the Translation task, which means that the policy is able to successfully infer the pose misalignment after just one or two failed attempts in most experiments. Given that the policies are purely trained with simulated tactile data, the high success rates indicate that our simulation is able to produce normalized tactile flow maps with highly realistic tactile patterns and magniudess to help the gripper to infer the exact adjustment. For challenging Rotation & Translation task, our zero-shot transferred policy also achieves 83% success rate and 4.81 pose corrections on average. For comparison, Dong et al. [10] achieve 89.6% success rate and 5.42 times pose adjustments for the cuboid object, with a policy trained directly on the real hardware from a pre-trained policy and with carefully designed task curriculum. On the other hand, our policy is trained from scratch only in simulation without observing any real-world data.
Limitations and Future Work
We presented an efficient differentiable simulator that can handle dense tactile force fields with both normal and shear components. When the tactile pad is very soft (e.g. TacTip), its dynamics cannot be well approximated by our penalty-based approach. An interesting direction explore is how to efficiently simulate such soft tactile sensors. We demonstrated with the box pushing task ($4.3) the potential advantage of differentiable simulators. However, how to effectively leverage analytical gradients for more complex tactile-based tasks is still an open question and it may require more advanced policy learning algorithms [40]. Furthermore, the zero-shot success rate of Rotation & Translation is not perfect. This is probably due to some intricacies of the real hardware that are difficult to model in our simulation. We believe that further fine-tuning the leamed policies with few shots on the real hardware will likely lead to improved performance.
the sim-to-real experiment ($4.5),

---

Acknowledgments
We thank the anonymous reviewers for their helpful comments in revising the paper. We also thank Toyota Research Institute (TRI), ONR MURI (N00014-22-1-- 1-2740., and the National Science Foun- dation (CAREER-I8460) for providing funds to support this work.
References
[1] E. Donlon, S. Dong, M. Liu, J. Li, E. Adelson, and A. Rodriguez. Gelslim: A high-resolution, compact, robust, and calibrated tactile-sensing finger. In 2018 JEEE/RSJ International Con- ference on Intelligent Robots and Systems (IROS), pages 1927-1934. IEEE, 2018.
[2] S. Sundaram, P. Kellnhofer, Y. Li, J.-Y. Zhu, A. Torralba, and W. Matusik. Learning the signatures of the human grasp using a scalable tactile glove. Nature, 569(7758):698-702. 2019.
[3] M. Lambeta, P.-W. Chou, S. Tian, B. Yang. B. Maloon, V. R. Most, D. Stroud, R. Santos, A. Byagowi, G. Kammerer, al. Digit: A novel design for a low-cost compact high-resolution tactile sensor with application to in-hand manipulation. JEEE Robotics and Automation Letters, 5(3):3838 3845, 2020.
[4] Taylor, S. Dong, and Rodriguez. Gelslim3. O: High-resolution measurement of shape, force and slip in compact tactile-sensing finger. arXiv preprint arXiv:2103.12266 2021.
[5] A. Church, J. Lloyd, R. Hadsell, and N. F. Lepora. Optical tactile sim-to-real policy transfer via real-to-sim tactile image translation. arXiv preprint arXiv:2106.08799 2021.
[6] M. Bauza, A. Bronars, and A. Rodriguez. Tac2pose: Tactile object pose estimation from the first touch. arXiv preprint arXiv:2204. 11701, 2022.
[7] E. Smith, R. Calandra, A. Romero, G. Gkioxari, D. Meger, J. Malik, and M. Drozdzal. 3d shape reconstruction from vision and touch. Advances in Neural Information Processing Systems, 33: 14193-14206 2020.
[8] S. Suresh, Z. Si, J. G. Mangelson, W. Yuan, and M. Kaess. Shapemap 3-d: Efficient shape mapping through dense touch and vision.
[9] R. Kolamuri, Z. Si, Y. Zhang, A. Agarwal, and W. Yuan. Improving grasp stability with rotation measurement from tactile sensing. 202/ JEEE/RS/ International Conference on Intelligent Robots and Systems (IROS), pages 6809-68 16. IEEE, 2021.
[10] S. Dong, D. K. Jha, D. Romeres, S. Kim, D. Nikovski, and A. Rodriguez. Tactile-rl for inser- tion: Generalization to objects of unknown geometry. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 6437-6443. IEEE, 2021.
[11] S. Kim and A. Rodriguez. Active extrinsic contact sensing: Application to general peg-in- in-lle insertion. arXiv preprint arX0v:2110.03555, 2021.
[12] W. Yuan, R. Li, M. A. Srinivasan, and E. H. Adelson. Measurement of shear and slip with a gelsight tactile sensor. In 2015 IEEE International Conference Robotics and Automation (ICRA), pages 304-311. IEEE, 2015.
[13] S. Dong, W. Yuan, and E. H. Adelson. Improved gelsight tactile sensor for measuring geom- etry and slip. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 137-144. IEEE, 2017.
[14] F. Veiga, H. Van Hoof, J. Peters, and T. Hermans. Stabilizing novel objects by learning to predict tactile slip. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5065-5072 IEEE, 2015.
[15] M. Li, Y. Bekiroglu, D. Kragic, and A. Billard. Learning of grasp adaptation through expe- rience and tactile sensing. In 2014 IEFF/RSJ International Conference on Intelligent Robots and Systems, pages 3339-3346. Icce, 2014.

---

[16] G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba. Openai gym, 2016.
[17] V. Makoviychuk, L. Wawrzyniak, Y. Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, et al. Isaac gym: High performance gpu based physics simulation for robot learning. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
[18] E. Todorov, T. Erez, and Y. Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference Intelligent Robots and Systems, pages 5026- 5033. IEEE, 2012.
[19] E. Coumans and Y. Bai. Pybullet, a python module for physics simulation for games, robotics and machine learning. 2016.
[20] O. M. Andrychowicz, B. Baker, M. Chociej. R. Józefowicz. B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, Ray, J. Schneider, Sidor, J. Tobin, Welinder, L. Weng, and W. Zaremba. Learning dexterous in-hand manipulation. The International Jour- nal of Robotics Research, 39(1):3-20, 2020.
[21] Hwangbo, J. Lee, A. Dosovitskiy, D. Bellicoso, V. Tsounis, V. Koltun, and M. Hutter. Learn- ing agile and dynamic motor skills for legged robots. Science Robotics, 4(26), 2019.
[22] J. Tan, T. Zhang, E. Coumans, A. Iscen, Y. Bai, D. Hafner, S. Bohez, and V. Vanhoucke. Sim- to-real: Learning agile locomotion for quadruped robots. In Robotics: Science and Systems, 2018.
[23] Xu, T. Du, M. Foshey, B. Li, B. Zhu, A. Schulz, and W. Matusik. Learning to fly: compu- tational controller design for hybrid uavs with reinforcement learning. ACM Transactions Graphics (TOG), 38(4):1-12, 2019.
[24] Z. Ding, Y.-Y. Tsai, W. W. Lee, and B. Huang. Sim-to-real transfer for robotic manipulation with tactile sensory. In 202/ JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 6778-6785. IEEE, 2021.
[25] A. Melnik, Lach, M. Plappert, T. Korthals, R. Haschke, and H. Ritter. Using tactile sensing to improve the sample efficiency and performance of deep deterministic policy gradients for simulated in-hand manipulation tasks. Frontiers Robotics and AI, page 57, 2021.
[26] S. Wang. M. Lambeta, P.-W Chou, and R. Calandra. Tacto: fast, flexible, and open-source simulator for high-resolution vision-based tactile sensors. Letters, 7(2):3930-3937, 2022.
IEEE Robotics and Automation
[27] Y. S. Narang, K. Van Wyk, A. Mousavian, and D. Fox. Interpreting and predicting tactile signals via a physics-base and data-driven framework. arXiv preprint arXiv:2006.03777, 2020.
[28) Y. Narang, B. Sundaralingam, M. Macklin, A. Mousavian, and D. Fox. Sim-to-real for robotic tactile sensing via physics-based simulation and learned latent projections. In 202/ IEEE International Conference on Robotics and Automation (ICRA), pages 6444-6451. IEEE, 2021.
[29] Z. Ding, N. F. Lepora, and E. Johns. Sim-to-real transfer for optical tactile sensing. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 1639-1645. IEEE, 2020.
[30] Z. Si and W. Yuan. Taxim: An example-based simulation model for gelsight tactile sensors. IEEE Robotics and Automation Letters, 2022.
[31] D. Ma, E. Donlon, S. Dong, and A. Rodriguez. Dense tactile force estimation using gelslim and inverse fem. In 2019 International Conference on Robotics and Automation (ICRA), pages 5418 1424. IEEE, 2019.
[32] T. Bi, C. Sferrazza, and R. D' Andrea. Zero-shot sim-to-real transfer of tactile control policies for aggressive swing-up manipulation. IEFE Robotics and Automation Letters, 6(3):5761- 5768, 2021.
10

---

[33] A. Habib, I. Ranatunga, K. Shook, and D. O. Popa. Skinsim: A simulation environment for multimodal robot skin. 2014 IEEE International Conference on Automation Science and Engineering (CASE), pages 1226-1231. IEEE, 2014.
[34] S. Moisio, B. León, P. Korkealaakso, and A. Morales. Model of tactile sensors using soft contacts and its application in robot grasping simulation. Robotics and Autonomous Systems, 61(1):1-12, 2013.
[35] J. Xu, T. Chen, L. Zlokapa, M. Foshey, W. Matusik, S. Sueda, and P. Agrawal. An End-to- End Differentiable Framework for Contact-Aware Robot Design. In Proceedings of Robolics: Science and Systems, Virtual, July 2021. doi: 15.07/.//..S....... 202..SS...2...I...0000
[36] Y. Wang, N. Weidner, M. A. Baxter, Y. Hwang, D. M. Kaufman, and S. Sueda. RED- MAX: Efficient & flexible approach for articulated dynamics. ACM Trans. Graph. 38(4), July 2019. ISSN 0730-0301. doo100.115533354433222222 URL httph: /tttps://doi...o org/10. 10. 1145/ 3306346. 3322952.
[37] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint 1107.v:777.063477 2017.
[38] M. Ahn, H. Zhu, K. Hartikainen, Ponte, A. Gupta, S. Levine, and V. Kumar. Robel: Robotics benchmarks for learning with low-cost robots. arxiv e-prints, page. arXiv preprint arXiv: 1909. / 1639, 2019.
[39] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel. Domain randomization for transferring deep neural networks from simulation the real world. 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 23-30. IEEE, 2017.
[40] J. Xu, V. Makoviychuk, Y. Narang, F. Ramos, W. Matusik, A. Garg, and M. Macklin. Accel- erated policy learning with parallel differentiable simulation. In International Conference on Learning Representations, 2021.


Reviews:
[{"review_id": "ablEsW2ARw", "reviewer": "Reviewer_BP21", "summary": "This paper presents a differentiable tactile simulator for robot manipulation. It implemented a penalty-based tactile model based on a differentiable RedMax to compute an analytical gradient for policy optimization. When performing the sim-to-real transfer, it proposes compensating the sim-to-real gap by seeing the sim and real tactile differences and normalizing the deformation.\n\nThis study performed various experiments to confirm speed, flexibility, and differentiability. The proposed simulation can run with 1050 FPS and perform various tasks and shapes of the tactile sensors. In the box pushing tasks, the proposed analytical gradient policy optimization showed faster convergence than the RL-based methods. Finally, it performed sim-to-real insertion tasks and demonstrated the zero-shot transfer.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "STRENGTH \n\nThis paper is excellently well written. The figures are beautiful. It made much effort to perform experiments in various environments. The experimental section is explained in detail. I believe this simulation must be helpful for users who are interested in tactile manipulation. The demo video is also beneficial to see if the simulation is close to the behaviors of the actual tactile sensors. I would highly appreciate it if this study released an open-source code for this simulation. \n\nWEAKNESS\n\nAlthough this study is excellent, it needs to address some minor points.\n\na) The meaning of \"efficiency\" sounds unclear. Which computational or sample efficiency does this study mean? If it means computational efficiency, this study needs more comparisons or discussions in section 4.1. How much faster is the proposed simulation than the existing simulators? Please provide quantitative explanations. Besides, I would recommend revising the manuscript entirely to clarify the efficiency. \n\nb) The proposed analytical gradient policy optimization demonstrated faster convergence in the box pushing. I would like to see the performance in the other tasks, such as insertion using the gradient method. This might be very important to support the usefulness of the proposed method.\n\nc) The soft robotics field is also exploring differentiable simulations. More discussions about differentiable models with soft robots might help enhance the novelty. The following reference might be an example.\n\nDubied, M., Michelis, M. Y., Spielberg, A., & Katzschmann, R. K. (2022). Sim-to-Real for Soft Robots Using Differentiable FEM: Recipes for Meshing, Damping, and Actuation. IEEE Robotics and Automation Letters, 7(2), 5015-5022.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Overall, this study is excellent. I would recommend weak acceptance for this paper.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "issues": "As described in the weakness, this paper needs to explain what \"efficient\" means in more detail. I would like to see whether gradient optimization can be used in different tasks.", "reviewer_expertise": "2: The reviewer is willing to defend the evaluation, but it is quite likely that the reviewer did not understand central parts of the paper"}, "extra_text": {}}, {"review_id": "Ty9Fdq7fABE", "reviewer": "Reviewer_VkwB", "summary": "The paper presents a novel differentiable physics simulator that includes differentiable tactile sensing simulation, and shows that it can facilitate Sim-to-Real Reinforcement Learning on robots equipped with GelSight tactile sensors.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n[1] The paper presents a novel differentiable tactile simulators for Robot Learning research.\n[2] Comprehensive experiments and its summary (i.e. what message does the experiment outcome represents) are presented.\n\nWeaknesses:\n[1] Please see comments and suggestions in the \"Issues\" section below, and please try to address them.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I recommend for a \"Strong Accept\" on this paper, given the paper's novelty and contributions, as well as comprehensive experiments and excellent presentation of the paper and results.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {"issues": "[1] Line 133-135 \"...which includes the tactile derivatives, is derived analytically, as shown below...\": I cannot seem to find the reason (on this paper) WHY do you need to derive the derivatives analytically, instead of relying on PyTorch’s auto-differentiation... Can the authors please elaborate?\n\n[2] In Equation (1) on Line 103-104, do you need to approximate the min() function with a softmin() function instead (for smoothness reasons and better differentiability)?\n\n[3] Line 140-141: I am a little lost when trying to follow the derivation that involves c (boldface c) in these lines. Could you please try to re-phrase to improve the clarity?\n\n[4] On Experiment 4.5, the authors show Sim-to-Real RL evaluation, but I cannot fully understand WHY it transfers quite successfully from simulation to the real hardware. Is it possible to add a more direct numerical comparisons between the data that the simulation produce as compared to the data that the hardware produce, and compare them side-by-side? So far, I only see in Figure 2 some pictorial comparisons between the data of sim vs real, but numerical comparisons would probably help in understanding better WHY the Sim2Real transfer was successful..."}}, {"review_id": "Ste_zd1osa", "reviewer": "Reviewer_Led2", "summary": "The paper develops a differentiable tactile simulator that models the tactile sensor with discrete points on the surface of the sensor and calculates the normal and shear force at these points from interaction with objects. The analytical equations used to compute these forces are differentiable, thereby enabling the use of gradients from tactile contact in downstream tasks. The paper uses this interaction function as a differentiable layer in pytorch and enables differentiation through dynamics. The paper runs experiments in simulation on a few different manipulation tasks and also runs object insertion experiments in the real world to validate the approach. ", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper does a good job in validating the proposed approach, especially compared to PPO (where there is a lack of gradients from tactile interactions). The variety of simulation experiments also help the readers in understanding the importance of gradients from tactile sensors.\n\nSome concerns I have with the paper are listed here: \n1. The paper mentions that existing FEM simulation models for tactile sensing (line 66) are slow which is false. Existing FEM simulation models are sufficiently fast while also giving high fidelity simulation of the contact area. The proposed method fails to simulate the surface deformation which is very common in robotic tactile sensors. Hence, the paper could instead mention that existing FEM simulation models lack gradients, preventing their use in gradient based optimization.\n\n2. Since the method models contact interaction with rigid body dynamics, how will this affect interaction with soft/deformable objects? Running some experiment in the real world to quantify this would be interesting. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "1. The paper’s approach cannot accurately simulate contact deformations, thereby making the method only work for tactile sensors that have a very stiff external surface.\n2. The paper’s simulation also cannot be used for pose estimation, which is a common application domain for tactile sensing as the simulation only reasons about forces at contact points.\n\n", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {"issues": "The main drawback of the paper is that the developed simulation is limited to only force based interaction tasks and also to sensors that have a near rigid external surface. Running a real world experiment with a soft/deformable object or using a softer tactile sensor would help in understanding the approach's limitation. A discussion on lack of contact field estimation from the simulation is needed in the limitations section. Especially discussing whether we can use the simulation for pose estimation tasks.\n\nIf the above limitations are in fact true, stating these in the abstract and changing the title to say \"rigid-body\" tactile simulation for manipulation would be helpful as readers could misunderstand the paper as being able to simulate all modalities from a tactile sensor. "}}, {"review_id": "NCRWmfbZne", "reviewer": "Reviewer_4x5o", "summary": "This paper presents a fast differentiable approach to vision-based tactile simulation that captures both the normal and shear tactile force measurements of tactile sensors of varying geometries. Experiments both in simulation and on the real robot showed that the analytical gradients provided by the tactile simulation accelerated the learning of different contact-rich tasks such as ball rolling, stable grasp adjustment, and peg insertion task. The speed of the tactile simulation (*1000 fps*) and impressive task learning performance suggests that this can be a very impactful work.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- The paper tackles an important challenge of simulating vision-based tactile measurement using a novel penalty-based model. The proposed approach not only captures the normal tactile forces as in a number of previous works, but also shear forces which has been expensive to compute until now\n- Impressively fast simulation speed was reported; achieving 1000 frames per second on a single core CPU.\n- Effective sim-to-real transfer of policies learned using the proposed tactile simulation method is very valuable. Given that many of the high resolution tactile sensors like GelSlim are pretty expensive, simulating them in a high fidelity way enables transferrable policy learning in simulation and can improve the useful life of the real sensors. The paper demonstrated zero-shot sim-to-real transfer of a policy learned entirely in simulation.\n- The paper is very well organized and clear. The derivations and details in the supplementary material aids the full understanding of the work. The video presents the experimental results in a very clear manner; tasks execution with overlaid tactile measurements were presented for the simulated task and real robot insertion task.\n\nWeaknesses\n- It is unclear how the simulation parameters $\\xi$ of the tactile point are determined for different tactile elastomer materials. This is an important detail for the reproducibility of the work either on the illustrated GelSlim sensor or for application to other tactile sensors.\n- The paper did not mention if the simulator will be released. Releasing the code will significantly increase the potential impact of the work.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper proposed a strong tactile simulation approach, and presented rigorous experiments to evaluate the fidelity and performance of the proposed methods. The ability to achieve zero-shot performance is impressive; this validates the fidelity of the simulator and showcases the potential impact of the work. ", "technical_quality": "Excellent: Very convincing and thorough.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {"issues": "- Some comments or discussion around selection of material parameters of the tactile sensor is needed. What's a good range and why? How can the parameter be determined for a different sensor e.g. DIGIT\n"}}]

Author rebuttal:

Decision: Accept (poster)