Paper: DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation (ICLR-2024)

Reviewer comment (Experiments):
While the accuracy of the optical sensor model is verified, the simulation itself would benefit from having a comparison to real-world experiments as well. Questions such as how well the dynamic behavior of soft objects match reality (bouncing objects), or how well the contact model applies to objects sliding/being pushed under friction. These would be more of a benchmark on the FEM-MPM-PBD simulation, but this could nevertheless broaden the applicability of the framework.

Author response:
Thank you for the thoughtful feedback. We have added real-world experiments to further evaluate our system in Section A.6.2. We also added the real2sim system identification demonstrations on our website to show the comparison between sim and real for pressing, sliding, and twisting scenarios. We believe simulating objects is essential for manipulation, several works such as Pac-Nerf [https://arxiv.org/abs/2303.05512], NCLaws [https://arxiv.org/abs/2304.14369] have presented the accuracy of soft object simulation with sim-to-real comparison. However, the focus of our work is the tactile sensor simulation and since our simulation system is modulized, it is straightforward to replace one or multiple modules with improved simulation approaches such as simulating objects with FEM or using constraint-based contact models. We would like to explore the fidelity of the different object simulation methods in future work.

Reviewer comment (Experiments):
How does the simulation compare when simulating articulated bodies? For example the lid-opening task, is the hinge a soft body or a joint? How well does this compare to modeling real-world objects?

Author response:
We approximate the articulated objects by using the MPM-based approach and assign different materials for different parts. Here the joint is simulated as a soft and thin body, and other parts are simulated as rigid bodies.  We further demonstrated that the optimized trajectory for the case opening task can be successfully deployed on real-world settings with a real articulated object in Section A.6.2.

Reviewer comment (Evaluation):
Adding some runtime reports, at least in the appendix, would be appreciated from a practitioner's perspective, since the choice for using FEM only for the sensor likely stems from high accuracy but high computational complexity as well. Hence MPM or PBD was used to simplify object simulation, is this correct? If so, it would be interesting to see how expensive each part of the simulation is, where simplifications are necessary when used in practice. Were any of the learned grasping or manipulation policies applied on the real robot?

Author response:
We have added the running time report in Section A.5 on all four contact-rich tasks. We provide both the simulation time and gradience backpropagation time, along with each simulation module (FEM, contact, MPM/PBD) running time in Table 11. 

And yes, MPM and PBD are used to simplify object simulation, we do observe fast PBD simulation speed but MPM simulation speed is rather slower because of the large amount of particles we used in simulation.

We also added two sets of real-world experiments to further evaluate our system in Section A.6.2. First we deploy the optimized trajectory from simulation on real-world robot setup for surface following and box opening tasks. Then we train a grasp stability prediction model in simulation based on tactile sensing data.  We directly apply the trained model to a real-world deformable object grasp task and show that we can adaptively grasp the soft deformable object.

Reviewer comment (Experiments):
Continuing on the topic of runtimes, for the trajectory optimization tasks in manipulation, it would be interesting to see how many iterations/computational resources each method was given to converge, was it the same for each, or was each method run until convergence?

Author response:
We have added the training curves of trajectory optimization on contact-rich manipulation tasks in Section A.4.6. To make a fair comparison with other baselines, we run all experiments with 100 iterations of optimization. From the training loss curves, we see most tasks can converge within 100 iterations.

Reviewer comment (Experiments):
Were the constraints applied to the reinforcement learning methods for trajectory planning also applied to the gradient-based optimization?

Author response:
Yes. Since our system is implemented with semi-explicit solvers which require a small simulation step size, we constrain the action space to [-0.15, 0.15] for RL and set small learning rates for gradient-based methods to ensure the stability of the simulation. And we use the same initial trajectories for both the gradient-based method and RL methods.

Reviewer comment (Presentation):
In Table 1, should the Tacchi method not also be differentiable since they are implemented in Taichi?

Author response:
It is true that Tacchi is implemented with Taichi, and it can be differentiable in theory. However, the authors of Tacchi didn’t implement the gradient backpropagation which is non-trivial and didn’t show any trajectory optimization with differential physics.

Reviewer comment (Experiments):
Are the results found from the SysID parameters used for the follow-up tasks?

Author response:
Yes, we use the identified tactile sensor parameters from Real2Sim for all following manipulation tasks. However, contact parameters such as the surface friction coefficient also depend on object materials. But these can still serve as good references and we use them by adding randomization based on the identified parameters. For object parameters, we randomize them within a range based on the tactile sensor's and contact model’s parameters to make sure the system can stably run.  All parameter details are added in the Section. A.4.5.

Reviewer comment (Evaluation):
In the system identification task, it seems like the pixel-wise tactile marker mean squared error is extremely high for the real to sim setting, or at least, the standard deviation or standard error (which one is it? It’s not labeled) is much larger than the differences between the different methods, including the random method. This is rather concerning, as it seems to indicate that the system identification method is not very effective at reducing the sim-to-real gap.

Author response:
We have labeled them as standard deviations. We admit the real-2-sim errors are higher than sim-2-sim which is normally because of the real-world sensor noise. Note that 1 pixel in the tactile image space roughly equals 0.113 mm in physical space, thus our real-2-sim is still at the submillimeter level. The mean squared errors are averaged over all markers from all frames, and we see the consistency of our lower errors compared to the baselines. We also observe the same scale consistency of standard deviations. Although standard deviations are higher than the differences between the different methods, these are attributed to the real-data noise level since our sensor is manually manufactured in the laboratory. 

We also added real-world experiments and demonstrated the trajectories optimized in simulation by using the real-2-sim results can be directly deployed on real-world setups. To further improve the real-2-sim results, collecting more data for system identification or using commercialized sensors can be explored as future work.

Reviewer comment (Evaluation):
Rather than a comparison between using or not using tactile sensing for grasping, which has been validated in prior works, or comparison between different methods for optimizing policies to solve the manipulation tasks, I wish the paper focused more on evaluating how realistic and accurate the tactile sensing simulation is, as well as sim to real applications. I think those are the things that will really impact whether or not practitioners can rely on this simulator to generate conclusions that will hold in the real world.

Author response:
We have added two sets of real-world experiments to further evaluate our system in Section A.6.2. First we deploy the optimized trajectory from simulation on real-world robot setup for surface following and box opening tasks. Then we train a grasp stability prediction model in simulation based on tactile sensing data.  We directly apply the trained model to a real-world deformable object grasp task and show that we can adaptively grasp the soft deformable object.

Reviewer comment (Experiments):
What differentiates the “grasping” task from the “manipulation” tasks? I think that the manipulation tasks are nice demonstrations of the types of tasks that can be simulated using DiffTactile, but I don’t quite understand what the “grasping” task is trying to show that is not being illustrated by the “manipulation” tasks, as it seems to me that it could easily become the fifth “manipulation” task.

Author response:
We have merged the “grasping” as one of the manipulation tasks. However since we have variations on the grasping task, we kept it stand-alone and renamed the rest four tasks as “contact-rich manipulation tasks”.

Reviewer comment (Presentation):
(nit): I recommend adding periods after paragraph section headings or otherwise distinguishing them from the following text.

Author response:
Thanks for the writing suggestions and we have added space between section headings and texts.

Reviewer comment (Writing):
(nit): For the “experimental results” on page 9: “For case opening and object reposing, we define the metric as the opened angle of the lid and the orientation of the object.” I think it’s not quite accurate to refer to the orientation of the object as a metric.

Author response:
Thank you for the writing suggestions, we have rephrased and clarified the sentence.

Reviewer comment (Evaluation):
It is difficult to judge the accuracy of system identification based on the MSE in tactile markers location alone. Small errors can lead to large drops in downstream task performance. Therefore, system identification algorithms are usually evaluated by sim2real task performance [1, 2]. This paper lacks sim2real task performance experiments.

Author response:
1. This paper lacks sim2real task performance experiments.

We have added two sets of real-world experiments to further evaluate our system in Section A.6.2. First we deploy the optimized trajectory from simulation on real-world robot setup for surface following and box opening tasks. Then we train a grasp stability prediction model in simulation based on tactile sensing data.  We directly apply the trained model to a real-world deformable object grasp task and show that we can adaptively grasp the soft deformable object.

Reviewer comment (Experiments):
Is CMA-ES not applicable to the system identification task (Section 4.1)? If it is, please discuss why it was not used as a baseline.

Author response:
2. Is CMA-ES not applicable to the system identification task (Section 4.1)? If it is, please discuss why it was not used as a baseline.

CMA-ES is applicable to the system identification task and we have updated Table 2 with CMA-ES results as baselines. From the results, we observed that CMA-ES performs second best after our proposed method in sim2sim scenarios, outperforming other baselines. However, its performance on real2sim is worse than RNN.

Reviewer comment (Evaluation):
A lack of discussion of computation time, especially the FEM-based deformation module.

Author response:
3. A lack of discussion of computation time, especially the FEM-based deformation module. What is the computational runtime of the proposed method, and how does it affect the intended applications?

We have added the running time report in Section A.5 on all four contact-rich tasks. We provide both the simulation time and gradience backpropagation time, along with each simulation module (FEM, contact, MPM/PBD) running time in Table 11. 

From the runtime reports, we show that the FEM module in the system can run greater than 30 fps for forward simulation, but the whole system’s speed also depends on object simulation and will double for the gradient backpropagation (backward). For example, the slowest task case opening requires 6 hours to optimize one trajectory with 600 simulation steps.

Reviewer comment (Reproducibility):
A lack of implementation details like RNN structure for parameter identification, optical prediction network architecture (Section 4.2), mathematical formulations of the reward functions used for the manipulation tasks (Section 4.4).

Author response:
4. A lack of implementation details like RNN structure for parameter identification, optical prediction network architecture (Section 4.2), mathematical formulations of the reward functions used for the manipulation tasks (Section 4.4).

We have added the RNN structure details in Section A.2. We also added the details of the reward functions in Section A.4.3 and A.4.4. Optical prediction network architecture will be added in Section A.3.

Reviewer comment (Evaluation):
What is the computational runtime of the proposed method, and how does it affect the intended applications?

Author response:
3. A lack of discussion of computation time, especially the FEM-based deformation module. What is the computational runtime of the proposed method, and how does it affect the intended applications?

We have added the running time report in Section A.5 on all four contact-rich tasks. We provide both the simulation time and gradience backpropagation time, along with each simulation module (FEM, contact, MPM/PBD) running time in Table 11. 

From the runtime reports, we show that the FEM module in the system can run greater than 30 fps for forward simulation, but the whole system’s speed also depends on object simulation and will double for the gradient backpropagation (backward). For example, the slowest task case opening requires 6 hours to optimize one trajectory with 600 simulation steps.

Reviewer comment (Reproducibility):
Which simulation parameters are used for the manipulation experiments in Section 4.4? Are they the parameters identified from the real robot system?

Author response:
5. Which simulation parameters are used for the manipulation experiments in Section 4.4? Are they the parameters identified from the real robot system?

We have added the simulation parameter details in Section A.4.5. Yes, we use the identified tactile sensor parameters from Real2Sim for all the following manipulation tasks. However, contact parameters such as the surface friction coefficient also depend on object materials. But these can still serve as good references and we use them by adding randomization based on the identified parameters. For object parameters, we randomize them within a range based on the tactile sensor's and contact model’s parameters to make sure the system can stably run.