Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper develops a differentiable tactile simulator that models the tactile sensor with discrete points on the surface of the sensor and calculates the normal and shear force at these points from interaction with objects. The analytical equations used to compute these forces are differentiable, thereby enabling the use of gradients from tactile contact in downstream tasks. The paper uses this interaction function as a differentiable layer in pytorch and enables differentiation through dynamics. The paper runs experiments in simulation on a few different manipulation tasks and also runs object insertion experiments in the real world to validate the approach. 
strengths_and_weaknesses: The paper does a good job in validating the proposed approach, especially compared to PPO (where there is a lack of gradients from tactile interactions). The variety of simulation experiments also help the readers in understanding the importance of gradients from tactile sensors.

Some concerns I have with the paper are listed here: 
1. The paper mentions that existing FEM simulation models for tactile sensing (line 66) are slow which is false. Existing FEM simulation models are sufficiently fast while also giving high fidelity simulation of the contact area. The proposed method fails to simulate the surface deformation which is very common in robotic tactile sensors. Hence, the paper could instead mention that existing FEM simulation models lack gradients, preventing their use in gradient based optimization.

2. Since the method models contact interaction with rigid body dynamics, how will this affect interaction with soft/deformable objects? Running some experiment in the real world to quantify this would be interesting. 

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: 1. The paper’s approach cannot accurately simulate contact deformations, thereby making the method only work for tactile sensors that have a very stiff external surface.
2. The paper’s simulation also cannot be used for pose estimation, which is a common application domain for tactile sensing as the simulation only reasons about forces at contact points.


issues: The main drawback of the paper is that the developed simulation is limited to only force based interaction tasks and also to sensors that have a near rigid external surface. Running a real world experiment with a soft/deformable object or using a softer tactile sensor would help in understanding the approach's limitation. A discussion on lack of contact field estimation from the simulation is needed in the limitations section. Especially discussing whether we can use the simulation for pose estimation tasks.

If the above limitations are in fact true, stating these in the abstract and changing the title to say "rigid-body" tactile simulation for manipulation would be helpful as readers could misunderstand the paper as being able to simulate all modalities from a tactile sensor.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: This paper presents a fast differentiable approach to vision-based tactile simulation that captures both the normal and shear tactile force measurements of tactile sensors of varying geometries. Experiments both in simulation and on the real robot showed that the analytical gradients provided by the tactile simulation accelerated the learning of different contact-rich tasks such as ball rolling, stable grasp adjustment, and peg insertion task. The speed of the tactile simulation (*1000 fps*) and impressive task learning performance suggests that this can be a very impactful work.
strengths_and_weaknesses: Strengths
- The paper tackles an important challenge of simulating vision-based tactile measurement using a novel penalty-based model. The proposed approach not only captures the normal tactile forces as in a number of previous works, but also shear forces which has been expensive to compute until now
- Impressively fast simulation speed was reported; achieving 1000 frames per second on a single core CPU.
- Effective sim-to-real transfer of policies learned using the proposed tactile simulation method is very valuable. Given that many of the high resolution tactile sensors like GelSlim are pretty expensive, simulating them in a high fidelity way enables transferrable policy learning in simulation and can improve the useful life of the real sensors. The paper demonstrated zero-shot sim-to-real transfer of a policy learned entirely in simulation.
- The paper is very well organized and clear. The derivations and details in the supplementary material aids the full understanding of the work. The video presents the experimental results in a very clear manner; tasks execution with overlaid tactile measurements were presented for the simulated task and real robot insertion task.

Weaknesses
- It is unclear how the simulation parameters $\xi$ of the tactile point are determined for different tactile elastomer materials. This is an important detail for the reproducibility of the work either on the illustrated GelSlim sensor or for application to other tactile sensors.
- The paper did not mention if the simulator will be released. Releasing the code will significantly increase the potential impact of the work.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The paper proposed a strong tactile simulation approach, and presented rigorous experiments to evaluate the fidelity and performance of the proposed methods. The ability to achieve zero-shot performance is impressive; this validates the fidelity of the simulator and showcases the potential impact of the work. 
issues: - Some comments or discussion around selection of material parameters of the tactile sensor is needed. What's a good range and why? How can the parameter be determined for a different sensor e.g. DIGIT
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper presents a differentiable tactile simulator for robot manipulation. It implemented a penalty-based tactile model based on a differentiable RedMax to compute an analytical gradient for policy optimization. When performing the sim-to-real transfer, it proposes compensating the sim-to-real gap by seeing the sim and real tactile differences and normalizing the deformation.

This study performed various experiments to confirm speed, flexibility, and differentiability. The proposed simulation can run with 1050 FPS and perform various tasks and shapes of the tactile sensors. In the box pushing tasks, the proposed analytical gradient policy optimization showed faster convergence than the RL-based methods. Finally, it performed sim-to-real insertion tasks and demonstrated the zero-shot transfer.
strengths_and_weaknesses: STRENGTH 

This paper is excellently well written. The figures are beautiful. It made much effort to perform experiments in various environments. The experimental section is explained in detail. I believe this simulation must be helpful for users who are interested in tactile manipulation. The demo video is also beneficial to see if the simulation is close to the behaviors of the actual tactile sensors. I would highly appreciate it if this study released an open-source code for this simulation. 

WEAKNESS

Although this study is excellent, it needs to address some minor points.

a) The meaning of "efficiency" sounds unclear. Which computational or sample efficiency does this study mean? If it means computational efficiency, this study needs more comparisons or discussions in section 4.1. How much faster is the proposed simulation than the existing simulators? Please provide quantitative explanations. Besides, I would recommend revising the manuscript entirely to clarify the efficiency. 

b) The proposed analytical gradient policy optimization demonstrated faster convergence in the box pushing. I would like to see the performance in the other tasks, such as insertion using the gradient method. This might be very important to support the usefulness of the proposed method.

c) The soft robotics field is also exploring differentiable simulations. More discussions about differentiable models with soft robots might help enhance the novelty. The following reference might be an example.

Dubied, M., Michelis, M. Y., Spielberg, A., & Katzschmann, R. K. (2022). Sim-to-Real for Soft Robots Using Differentiable FEM: Recipes for Meshing, Damping, and Actuation. IEEE Robotics and Automation Letters, 7(2), 5015-5022.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: Overall, this study is excellent. I would recommend weak acceptance for this paper.
issues: As described in the weakness, this paper needs to explain what "efficient" means in more detail. I would like to see whether gradient optimization can be used in different tasks.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The paper presents a novel differentiable physics simulator that includes differentiable tactile sensing simulation, and shows that it can facilitate Sim-to-Real Reinforcement Learning on robots equipped with GelSight tactile sensors.
strengths_and_weaknesses: Strengths:
[1] The paper presents a novel differentiable tactile simulators for Robot Learning research.
[2] Comprehensive experiments and its summary (i.e. what message does the experiment outcome represents) are presented.

Weaknesses:
[1] Please see comments and suggestions in the "Issues" section below, and please try to address them.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I recommend for a "Strong Accept" on this paper, given the paper's novelty and contributions, as well as comprehensive experiments and excellent presentation of the paper and results.
issues: [1] Line 133-135 "...which includes the tactile derivatives, is derived analytically, as shown below...": I cannot seem to find the reason (on this paper) WHY do you need to derive the derivatives analytically, instead of relying on PyTorch’s auto-differentiation... Can the authors please elaborate?

[2] In Equation (1) on Line 103-104, do you need to approximate the min() function with a softmin() function instead (for smoothness reasons and better differentiability)?

[3] Line 140-141: I am a little lost when trying to follow the derivation that involves c (boldface c) in these lines. Could you please try to re-phrase to improve the clarity?

[4] On Experiment 4.5, the authors show Sim-to-Real RL evaluation, but I cannot fully understand WHY it transfers quite successfully from simulation to the real hardware. Is it possible to add a more direct numerical comparisons between the data that the simulation produce as compared to the data that the hardware produce, and compare them side-by-side? So far, I only see in Figure 2 some pictorial comparisons between the data of sim vs real, but numerical comparisons would probably help in understanding better WHY the Sim2Real transfer was successful...
review_title: null

Meta-review: metareview: This paper presents a differentiable and fast contact model including shear force that enables efficient reinforcement learning with simulated tactile sensors of arbitrary shapes. The model is used with PPO to learn 3 tasks in simulation. The authors also present sim-to-real transfer in the insertion task.

The paper is well written and easy to follow, and simulation and hardware evaluations are convincing. The authors' responses successfully addressed the concerns such as comparison to FEM-based methods.

While the main contribution (analytical derivative of a contact model) is useful beyond robot learning, it is a tool that also benefits this community as demonstrated by its applications to RL in the paper.
confidence: 5: The area chair is absolutely certain
Decision: Accept (Poster)