Venue: CoRL 2021 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper develops a learning paradigm to obtain non sheared tactile images from shear contact.
strengths_and_weaknesses: The paper is attempting to solve an interesting problem which benefits the domain of visuotactile sensors for tactile perception. The illustrations in figures bring clarity to the approach. There are some concerns with the paper in its current form which are listed below:

1. One major concern with the approach is that it’s not clear to me whether the approach works on unseen objects. Also, it was not clear whether the learned model is object dependent. Conducting experiments on unseen objects would help bring clarity.

2. Does the model generalize across sensors or do we need to train per sensor?

3. Additionally, the paper mentions an appendix but it was not submitted. 


summary_of_recommendation: I would like to know whether the model generalizes before I can make a better recommendation.
issues: Addressing the below issues will improve the paper:

1. It’s not clear whether the model was trained separately for each object or if a single model was trained across objects.

2. Running experiments on unseen objects.
review_title: null

final_score: rating: Weak Reject: I recommend rejecting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper proposes a supervised learning method to remove artifacts from optical-tactile signals for robot manipulation tasks. Specifically, the artifacts in question are caused by sensor distortion in the presence of motion-induced shear. In the approach, the model is trained on tactile images collected from un-sheared contact with a few distinct canonical shapes. 
strengths_and_weaknesses: Strengths:
- The paper is generally well written and easy to follow
- Experiments are detailed and the approach is clearly defined.
- Results seem promising.

Weaknesses
- The scope of the paper strikes me as fairly narrow, both in experiment and comparison to existing methods.
- Why does the approach not consider task-specific training, such as predicting local surface geometry directly for surface mapping (as opposed to an intermediate un-sheared representation) ? It would be nice to highlight the benefits of this approach when compared to approaches which model local contact geometry, such as the following (I have no affiliation):
- Bauza, Maria, et al. "Tactile object pose estimation from the first touch with geometric contact rendering." arXiv preprint arXiv:2012.05205 (2020).
- There seems to be a lack of baselines in the approach. How might this compare to using an analytical model, for instance? 
- The scope of the experiments seem quite narrow. Only three shapes were investigated, and all were made from the same material. To demonstrate that the method is indeed learning a meaningful representation (as opposed to over-fitting to data), it would be useful to use test objects and motions which were not observed in training. 
- There does not seem to be any discussion of rotational vs translational shear. I would think that the former would be more challenging to disentangle than the latter, but this should be discussed and investigated.
-  A description and picture of the TacTip sensor would be very helpful to the reader.

summary_of_recommendation: A significant contribution to the community would require broadening the scope of the paper, clearly demonstrating the generality of the approach comparison to existing state-of-the-art methods.
issues: Please address the points listed above.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper presents a method for handling the motion-dependent shear effect in tactile sensors. In particular, as objects can move under soft tactile sensors, tactile signal is distorted or sheared, which leads to difficulties in employing the resulting tactile signal, for example when performing object pose estimation. The paper proposes to learn “unshearing” deep neural network models that can map from sheared tactile images to canonical non-sheared images. Two architectures are proposed: entangled architecture that directly learns to reconstruct the canonical image from the sheared one, and a disentangled architecture that decomposes the intermediate representation into canonical and shear-specific parts that are trained to reconstruct the canonical and the original sheared tactile images respectively. Finally, the unsheared images are used to estimate object poses using a pose-model trained on the canonical non-sheared images. The paper collects a paired dataset of sheared and canonical tactile data to train the unshearing models. The resulting models are evaluated on three objects and TacTip biomimetic tactile sensor. They are shown to produce unsheared tactile images with a high similarity to the canonical ones and successfully reconstruct the original geometry of the objects, as well as significantly improve object pose prediction, with the disentangled model having a higher performance than the entangled one. Pose prediction is also used to perform servo control of robot following contours of the objects.
strengths_and_weaknesses: Strengths:
- Reducing the shearing effect of tactile sensors is important for real-world application of tactile signals.
- Disentangling reconstruction into sheared and non-sheared parts is an elegant way to improve unshearing performance.
- The paper is well-written and easy to understand and follow.
- The method is shown to work on a real TacTip biomimetic tactile sensor and produce a high level of reconstruction accuracy.

Weaknesses:
- Although the dataset is split into training and test parts, all the data still comes from the same three objects. It would be useful to see performance of the method on some novel objects to better understand the generalization capability of the method.
- Right now, the PoseNet is trained on canonical images and evaluated on sheared/unsheared images. It would be interesting to see results of training the PoseNet directly on the sheared images if such a pairing is possible. This would help to better understand the advantage of reconstructing the unsheared image for pose prediction. 
- It would be interesting to see more details on the robotic control system used to perform the contour following experiments.
summary_of_recommendation: The paper shows that it is possible to unshear tactile sensor signals, which is important for many real-world robotic applications. The paper could benefit from some additional experiments to demonstrate the generalization capabilities of the method.
issues: Please refer to "Strengths And Weaknesses" for a detailed list of issues.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This work proposed to disentangle the contact geometry from shear, which can benefit sliding motion and manipulation for soft tactile sensors. The authors proposed the encoder-decoder network trained from paired supervised data. The system is evaluated to reconstruct unsheared canonical images, local shape reconstruction. It is further applied for contour following with sliding motion.

strengths_and_weaknesses: Strengths:
- The disengagement of the contact geometry and shear is meaningful, and can benefit future applications with shear-related tactile and manipulation tasks.
- The author proposed a network and automatic data collection framework to tackle the problem, using paired supervised data.
- The disentangled results are evaluated for contour following.

Weaknesses:
- The comparison between Entangled and Disentangled UnshearNet seems unfair. The Disentangled UnshearNet clearly has more supervision, in the autoencoder fashion to reconstruct the raw image. A fair comparison could be also adding raw input reconstruction to the Entangled network. By current comparison, it is unclear whether the improvement is from the additional supervision or the disentangled network flow.
- Additionally, there seems to be no clear evidence that the embedding of the Disentangled network is disentangled into “Pose Code” and “Shear Code”. What if the “Pose Code” contains both “deformation” and “shear” information, and “Shear Code” contains minor/part of shear information? It seems that the network flow doesn’t have the supervision to force the embedding to be disentangled, since the “Pose Code” is used for both geometry and raw input reconstruction. And there is no validation of what the “Shear Code” learns. One experiment would be using the “Pose Code” and “Shear Code” to predict the motion that induced the shear. It can also be added as the supervision to the network to force “Shear Code” alone to learn the motion. 
- For 3.2.2 Pose-labelled data, the data collection procedure seems very similar to the one for canonical data. Is there a reason for collecting a separate dataset?

summary_of_recommendation: Overall, the paper demonstrated an important problem of entangled shear for contact geometry. The framework with the supervised paired data solved the problem well. But the technical part of the network structure requires more experiments to support its claim of disentanglement of “Pose Code” and “Shear Code” for the Disentangled UnshearNet.

issues: Quote from the weaknesses:
- The comparison between Entangled and Disentangled UnshearNet seems unfair. The Disentangled UnshearNet clearly has more supervision, in the autoencoder fashion to reconstruct the raw image. A fair comparison could be also adding raw input reconstruction to the Entangled network. By current comparison, it is unclear whether the improvement is from the additional supervision or the disentangled network flow.
- Additionally, there seems to be no clear evidence that the embedding of the Disentangled network is disentangled into “Pose Code” and “Shear Code”. What if the “Pose Code” contains both “deformation” and “shear” information, and “Shear Code” contains minor/part of shear information? It seems that the network flow doesn’t have the supervision to force the embedding to be disentangled, since the “Pose Code” is used for both geometry and raw input reconstruction. And there is no validation of what the “Shear Code” learns. One experiment would be using the “Pose Code” and “Shear Code” to predict the motion that induced the shear. It can also be added as the supervision to the network to force “Shear Code” alone to learn the motion. 
- For 3.2.2 Pose-labelled data, the data collection procedure seems very similar to the one for canonical data. Is there a reason for collecting a separate dataset?
review_title: null

Meta-review: This paper presents a machine learning model to disentangle the effects of motion-induced shear from contact geometry in vision-based tactile sensors.

Quality:

(-) All reviewers have concern that the experimental validation does not fully support the claims of the manuscript. I here highlight the concerns from Reviewer gcAq and JxBB regarding the generalization capabilities of the proposed approach to unseen objects and sensors, and the additional experiments suggested by Reviewer xr1d to validate the claims about the Disentangled UnshearNet. Performing the experiments requested by the four reviewers would benefit the overall quality of the paper. 

(+) Real-world experiments

Clarity:

(+) The paper is generally clear and well written

(/) The appendix is not included

Originality:

(+) The proposed approach is novel 

Significance:

(+) Several reviews agree that the topic is relevant and impactful

Additional comment: From a philosophical perspective, it should be better argued why Disentangling Contact Geometry from Motion-Induced Shear is useful. If you consider that this is a component of a wider system (e.g., control), a devil's advocate might ask why not using an end-to-end learning approach (together with the downstream task) in the first place and side-step the need to explicitly disentangle.


---

After rebuttal:
The authors included additional experiments and answered the reviewers in such a way as to address most of the concerns.
While the paper is likely to impact a very niche part of the robotic community, the contribution appears to be solid.
Decision: Accept (Poster)