Venue: CoRL 2021 Conference

Reviews:

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The paper proposes an approach for simulating optical tactile sensors using depth imaging and approximation with rigid body dynamics. Subsequently, real2sim image translation is utilized to enable policy transfer from simulation to reality. The approach is validated on a suite of tactile RL environments such as edge/surface following and object rolling/pushing/balancing.
strengths_and_weaknesses: Strengths: the paper proposes a novel and practical approach to bridging the sim2real gap when using optical tactile sensors. The idea of using image translation appears straightforward and useful. The evaluation is thorough, and the source code is provided. The proposed method can be applied to a whole family of optical tactile sensors, making it interesting to a broader community. The suit of tasks proposed in the paper can serve as a basis for future tactile control benchmarks.

Weaknesses: writing can be improved
i) Provide a succinct and self-contained description of the complete approach, e.g., via a diagram or an algorithm box. One can reconstruct roughly what the algorithm is but it would help the reader to see it at once and formulated in a concrete manner.
ii) In general, sections seem hanging independently and there is a lack of cohesive narrative in the paper. It reads a bit like a lab report. Adding some connecting sentences and an overarching narrative would improve the flow.
summary_of_recommendation: The idea of simulating tactile features using depth images and rigid body dynamics with subsequent image translation is proposed and evaluated. Tactile non-prehensile manipulation tasks are used to evaluate the method. The idea is good and the evaluation is well-executed. The results suggest that the method is viable. Limitations and directions for future work are discussed in detail. This paper may serve as the basis for further investigations into simulation and control based on optical tactile sensors.
issues: See weaknesses above
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes an efficient method to simulate tactile images and show that RL policies trained within the simulation can transfer to real-world settings with an learned image-to-image translation module that approximately turns real-world image into simulated image. The results are demonstrated on several tasks defined by the authors.

strengths_and_weaknesses: **Strengths**: 
- The paper is well-written and organized. Each section has very clear paragraphs that summarize both the motivations and design decisions.
- The sim-to-real results are encouraging. As collecting tactile images can be very time-consuming in the real world, successful sim-to-real results can encourage more people to explore this direction.
- The released simulation and benchmark can be very useful for the community.

**Weaknesses**: 
- (Nitpicking) no major novelty is proposed. Image-to-image translation for sim2real has been explored in more complicated domains (e.g., vision), so showing that the same technique works for tactile is generally less surprising.

summary_of_recommendation: I recommend accepting the paper because to the best of my knowledge, this is the first paper that presents results on successful sim-to-real tactile-based policy with *domain adaptation*.  Additionally, the paper is well-written and the experiments show real-world results.

issues: - May authors clarify what does blue represent in Figure 6? It’s not shown in the legend.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: In this work, the authors propose a method for successful sim-to-real transfer of manipulation policies utilizing tactile perception for a common type of optical tactile sensors. More specifically the authors present: 
- A simple and fast simulation method for tactile sensors, where contact geometry is represented as depth images;
- Data driven algorithm to translate current state of a real tactile sensor to state of tactile sensors in simulation.
- Successful results of learning various manipulation policies utilizing tactile perception in simulation.
- Successful zero-shot sim-to-real transferability of the learned policies in majority of tasks.

strengths_and_weaknesses: Strengths:
- The paper is well organized. It is interesting to read and simple to follow.
- The paper tackles problems on the intersection of two important topics: tactile perception and sim-to-real transfer. 
- The paper proposes a method that can find practical applications in various real world settings. Successful results strongly support the applicability of the approach.

Weaknesses:
- It seems that the method would work well with just certain types (albeit a quite popular set) of tactile sensors. It will likely fail with more complex sensors, such as BioTac. Overall, it is important to demonstrate what specific sensors it can and cannot work to see the limitations of the approach. 
- The method requires a separate procedure to learn real-to-sim mapping of the sensory data models. It is a bit hard to grasp from the description in the paper how complicated is the setup for such data collection? 
- As far as I understood, the paper does not strongly demonstrate sim-to-real generalization toward strong variation of the tasks themselves. Can it follow new contours and edge shapes? Push various types of objects? Roll other objects (at least various sizes). It does seem like the simulated scene of the setup should match parameters of the actual experiment, which may, in some cases, limit applicability of the approach.

summary_of_recommendation: Despite moderate scientific contribution and not overwhelming scope, this is a well written and interesting work that can find its applications. I am recommending “weak accept”.
issues: The authors mentioned a possible cause of oscillations and several potential solutions to eliminate action range clipping in the policies. It would be interesting to have an investigation of these effects, as well as more extensive investigation of generalization capabilities of the approach.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: This paper applies sim2real RL policies for several real-world robotic tasks that benefit from tactile sensing. The authors show that PPO can successfully solve the tactile tasks in simulation. To bridge the sim2real gap for policy transfer, the authors apply real2sim using GAN, which transforms complex marker arrays to simulated contact depth. Evaluation is done in both simulation and real environments. The authors plan to open-source the tactile simulator and environment for TacTip, which can benefit future research. 
strengths_and_weaknesses: Strengths:

- The paper is clear and easy to follow.
- It is novel to apply GAN to bridge the real2sim gap for tactile signals. The authors put extra consideration to disentangle the influence of shear force for better real2sim.
- It is appreciated for evaluating the RL policy for both simulated and real environments and comparing the performance of different modalities in simulation.
- Supplementary video demonstrates well the performance of the system in simulation & real environments.
- The codes are planned for open-sourcing.

Weaknesses:

- For the proposed tasks, feeding a low-dimensional contact state (like contact center position and major contact orientation) combined with simple PID controllers seems to be sufficient enough to solve the tasks. The whole pipeline of RL and GAN seems to be solving simplified problems. 
- Among the tasks, the pushing task is the most challenging and interesting one, since it has to deal with objects with unknown dynamics. But there is no clear evidence that the proposed method is better than any baseline or intelligent/robust to uncertainties in the real world scenario. It is suggested to add some disturbance to study how robust the system is.

summary_of_recommendation: The authors put good effort to push towards benchmark and sim2real policy transfer for tactile robotic tasks. It’s novel to apply GAN for tactile signals to bridge the real2sim gap, and the framework can potentially benefit many future works. Evaluations on simulation and real-world are appreciated. It can be further improved with some more challenging tasks, comparing the system with baseline control policies in the real world, and studying how intelligent/robust the system is under disturbances.

issues: As stated in the Weaknesses:
- For the proposed tasks, feeding a low-dimensional contact state (like contact center position and major contact orientation) combined with simple PID controllers seems to be sufficient enough to solve the tasks. The whole pipeline of RL and GAN seems to be solving simplified problems. 
- Among the tasks, the pushing task is the most challenging and interesting one, since it has to deal with objects with unknown dynamics. But there is no clear evidence that the proposed method is better than any baseline or intelligent/robust to uncertainties in the real world scenario. It is suggested to add some disturbance to study how robust the system is.
review_title: null

Meta-review: This paper focuses on the problem of sim-to-real for tactile sensing.

Clarity: The paper is generally clear as mentioned by all the reviewers, but as pointed out by Reviewer 3URa the paper could benefit from a bit more polishing. In particular, when it comes to Sec. 5, and how it fits in the general scheme (Reviewer 3URa and 3knc).

Quality: The real-robot experiments are certainly well appreciated by the reviewers, as well as the promise to open-source the code. The work overall has several interesting bits, but the manuscript seems to claim a lot of contributions while failing to thoroughly investigate any of them. To me, it would seem that the main (and more valuable) contribution could be to show how the Real-to-Sim Tactile Image Translation provides superior performance for sim2real transfer. However, the current manuscript does not provide any experiment that compares trivial sim2real to the proposed approach. As such, it is impossible to understand how much of the sim2real performance of Real-to-Sim Tactile Image Translation is due to the simulator being accurate vs the use of the proposed approach. (The simulator itself is also listed as a contribution, but is not evaluated in any way)
In my opinion, adding this experiment and being a bit more precise regarding the actual contributions would improve the manuscript. 

Originality: I find the novelty of the paper rather limited. As mentioned in the related work there are several tactile sensing simulators already existing, and the idea of sim-to-real for tactile sensing has also been previously investigated. The paper does not compare to any of them. The algorithm proposed is on itself also adapted with minor changes from the computer vision community, as pointed out by Reviewer 7RPs.

Significance: Sim2real for tactile sensing is certainly an important and timely topic. While this is not the first paper dealing with the problem, a strong demonstration that the proposed method improved over trivial sim2real and/or existing methods would be valuable to the community.

Other comments:
- Related works do not currently provide an accurate depiction of the existing literature in tactile sensing simulators and sim2real, and as such should be improved. Related works in sim2real for tactile sensing should be explicitly discussed (some are already cited in other parts of the paper).
- In the experiments of Fig.3 the fact that the Oracle does not always provide the best performance is troubling, and I am not convinced by the explanation of Sec. 4.3. To me, this seems to indicate either that the Oracle is not providing sufficient information (hence it shouldn't be called Oracle in the first place), or that the learning algorithms are not properly tuned.

Decision: Accept (Poster)