Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The paper presents an approach for controlled pivoting of objects using tactile sensing. The outcomes of an LSTM approach are compared to an MLP based approach and shown to be superior. Vision is used as a ground truth.
strengths_and_weaknesses: Strengths

The problem of controlled gravity based pivoting has been addressed by several other papers recently and so is a current topic of interest in robotics research..

Weaknesses

The MLP approach that the system is compared against is not well described and motivated in the main body. Though the parameters of the MLP are given in the supplemental material, it is not clear how these were determined and why the approach of [5] was selected as inspiration. Given the key antagonistic role the MLP approach plays in this work, the authors must convince the reader that they didn't intentionally design / tune a system to have poor performance. 

The paper is unclear at times and it took me some backtracking to make sense of the results.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The paper is generally interesting and the arguments for the reason for investigating the problem are compelling. My major qualm with the paper comes from the MLP comparison method, which, as mentioned in the weaknesses, I think needs more text to justify why it was selected (compared to the approaches of oether papers) and how it was tuned. Until this is done, the outperformance claim is somewhat weak. 

I have listed some minor issues in the following section which affect paper clarity. I mention this here as there were quite a few of these, which contributed to a sense of frustration by the time I finished reading the manuscript.  
issues: * In the first sentence of the introduction - it seems that such general claims on robotic manipulation would be better served by citing review papers rather than these specific pieces of work. 

* It seems that the following paper from Bristol would be a relevant addition to the literature review - 'Slip Detection With a Biomimetic Tactile Sensor' 2018.

* The motivation for adding masking tape to the objects was not explained - we are simply told that friction is augmented. I assume this is to make the surface friction consistant across the objects (which simplifies the pivoting problem somewhat). Could the authors also make it clear that this was applied to all trials. I first read this as the tape being a variable.

* Why are two of the objects covered in green paper but the others are not?

* Are the objects full or empty? For example the deoderant and pills.

* We are not told how much control is possible over the gripper, in terms of grasp aperture resolution. It would be interesting to know what is the smallest amount you can reliable move one of these grippers and how much your controller has to move it. 

* How are the 2x9 papillae of the tactile sensors combined to get a force output? Or are all 18 ouputs fed into the models?

*  I could not make sense of Table 1 for sometime until I read in Section 7.2 that the headings were defined in section 6.2. Though I had read section 6.2, I did not recall the described labels when seeing the acroynm, as there was quite a bit of space between 6.2 and the table. There is no harm in reminding the reader in the caption of the table and using the same acroynms elsewhere, such as on Figure 5. 

* The benefits of slower rotational motion is argued in Section 8, but a counter-argument would be that this would slow down any processes that the robot is engaged in. An example is given earlier in the paper of shelf stacking, which would be a repetitive industrial task where speed is prized. 

* I think a current limitation of the method is the reliance on the tape to normalise the frictional properties of the objects and would like to see this discussed in the limitations section.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This work studied pivoting objects using gravity with tactile sensors. It proposed a learning-based method to track the angle and velocity of the object in hand using force-based tactile sensors. The tracking model is used in a closed-loop control to pivot the object to a target angle. The proposed LSTM model outperforms the baseline MLP models in the tracking and closed-loop control tasks. In addition, the experiments show promises of transferring to unseen objects.
strengths_and_weaknesses: Strengths:
- It is an important topic of in-hand manipulation that can benefit from tactile sensors.
- The proposed models achieve promising results and outperform baselines in perception and control tasks.
- The held-out experiments also show promises to generalization to unseen objects.
The paper is well written and easy to follow.

Weaknesses:
- The comparison between MLP and LSTM seems unfair. The MLP only has the last 15 timestamps, but LSTM has access to longer sequences. It is unclear whether more data or the better model causes the improvement. It will be clearer if there are experiments when the LSTM model also uses the last 15 timestamps.
- As the author mentioned in the Limitation section, the controller is relatively simple. The model requires relatively large movements to observe, which will introduce rather large errors. The author could consider comparing different controllers to achieve better performances.
- The physical properties of the objects should be described, like sizes, mass, center of mass, and surface roughness. This helps to interpret the data distribution, and see the generalization performance to different physical properties.
- The author should also discuss the range of suitable objects for this method. Light objects might be challenging to pivot based only on gravity.
- The work can be improved with video demonstrations to show tactile signals, real-time tracking, and the control policy.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This work provided a learning-based method for gravitational pivoting with real-time tactile feedback. The proposed models showed promising results and outperformed the baseline model. The perception result was further evaluated with a controller for application. While overall is good, it could further be improved by exploring more advanced controllers, models, fair comparison between baselines, and more details about the generalization.
issues: As discussed in Weaknesses.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper leverages gravity to achieve in-hand manipulation of prism-like objects via pivoting objects to a desired angle. The proposed solution has two components: (1) a Rotational Slip Estimator LSTM (RSE-LSTM) that estimates the relative angle change and velocity of the object based on tactile information and (2) a grip controller that uses RSE-LSTM to open the gripper until the target angle for the object is achieved. Since the control is based on the feedback of a learned tactile module, the system assumes no knowledge of object parameters, other than the fact that the object is prism-like and will thus experience minimal translational slip relative to the rotational motion. For the RSE, an LSTM-based model outperforms an MLP-based model on unseen objects. It is also shown that it improves performance to predict angle and velocity. The LSTM model leads to better controller performance, (compared to MLP), although even with fine-tuning both lagged behind the oracle's performance.
strengths_and_weaknesses: This paper prevents a novel method for pivoting an object in-hand in order to modify the object's orientation. In contrast to many other in-hand manipulation systems, the system does not assume any knowledge of object properties, which should lead to increased generalization. The issue of accurate and very dynamic manipulation, as tackled in this paper, is a tricky one. As stated in the limitations, the relatively simple controller serves as a good jumping off point for future work.

The paper is well-written, with the structure clearly laying out each contribution. Section 3 was particularly helpful in clearly stating the problem and the guiding assumptions. The experimental evaluation section was comparatively extensive. Given that the closest related work used an MLP in the context of translational slip, it made sense to compare LSTM and MLP.

As stated above, accurate and dynamic manipulation is tricky. However, it would strengthen the paper to better understand what could be done to improve the generalization to new objects (particularly since generalization is a strength of the approach). Is this primarily a weakness of the estimator or the controller? The paper mentions that some objects are harder to generalize to. What are the properties of these objects that contribute to this?

It would also be great to include some videos of the system in action as part of the supplementary material. 
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I am weakly recommending the acceptance of this paper. The paper presents an interesting in-hand manipulation method that leverages tactile sensing to perform dynamic pivoting. The formulation does not rely on any object properties (aside from being prism-shaped) and thus can generalize to many objects, although there may be some more tweaks or training needed to fully fulfill this promise. The paper is well-organized and provides appropriate in-hand manipulation demonstrations.  

issues: As stated above it would be great to have a bit more discussion (perhaps in the appendix, since the paper is space constrained) about how to improve the generalization to new objects (or what is a limiting factor preventing this). This could include discussion of what properties of objects make generalization difficult. For example, there is a comment in Section 6.4 that error for the toothpaste is higher. What about the toothpaste causes this?

In collecting the dataset, there were two methodologies for controlling the gripper, "Rotate to Stop" and "Angle Goal". Was there a significant difference between the data collected by these two methodologies? Would there be an impact in using data gathered by one over the other? Additionally, I believe the "Angle Goal" controller leverages the RSE-LSTM, which is trained based on data collected by this process. It would be helpful to clarify this.

In algorithm 1, is there any mechanism that prevents the gripper from opening past the point where the object would fall out? Would it be possible to integrate such a mechanism?

If possible, videos of pivoting the various objects (particularly the unseen objects) would strengthen the paper. 

These two related papers might be worth discussing and citing: 
- Sintov, Avishai, and Amir Shapiro. "Swing-up regrasping algorithm using energy control." In ICRA, pp. 4888-4893. IEEE, 2016.
- Holladay, Anne, Robert Paolini, and Matthew T. Mason. "A general framework for open-loop pivoting." In ICRA, pp. 3675-3681. IEEE, 2015.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper looks at the in-hand manipulation of objects with a two fingered gripper by relaxing the grip to allow the object to rotate under gravity. A simple 2 state controller is used (slightly open / hold) depending on the object's angle and rate of change of angle, aiming to reach a desired angle of rotation. These are estimated using a MLP and LSTM on tactile data from two PapillArray sensors mounted on a Robotiq parallel-haw gripper. The networks are trained on several objects and tested on novel objects. Overall, the LSTM works best and is effective with an angle error of 5-10deg. 
strengths_and_weaknesses: Strengths 
- Fairly novel task for tactile sensing. Could be used as a benchmark to compare sensors, grippers etc.
- Nice combination of machine learning and control.
- I learnt that you also need to use the rate of change of angle when trying to reach a desired angle, as a correction on the latency in the system.

Weaknesses
- The controller has just two preset states (close or slightly open). How is that tuned to the different object sizes? Is that done manually? As you have a tactile sensor it would have been nice to use the sensor to set how closed and open you need to be.
- Prediction of object angle is a basic capability that there are many ways of doing with different sensors (e.g. using image moments etc). What made your particular sensor well suited for this? I had no sense of how the tactile data was interpreted by the ML method to make the prediction of angle.
- The scope seemed limited. Once the method was shown to work, it would have been good to see some other demonstration of its capability or more explanation of aspects of it to broaden the scope. E.g. rather than gravity you could use translate the gripper so the object is turned by knocking against something.
- You could have been more complete in your referencing - e.g. there are plenty more on slip and related topics (including a review article by the inventors of the sensor in this paper). Not a major weakness, but I don't think there is any limit on referencing at this conference.
- How suitable is the sensor for this task. As I understand it, it has a relatively low dimensional output (3 x 3 array) that is well suited for slip detection. However, you are using it for object pose estimation, where I would have thought more detailed sensors e.g. the DIGIT would be more effective. I would have liked to have seen some benefits of the tactile sensor you are using coming through in this study. 
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I like the paper, but veer towards a weak accept because the scope is narrow and the problem is somewhat straightforward to solve. I also did not learn much about how the tactile sensor data is intererpreted to allow the task to be succesful, which would have been interesting.  
issues: Please see comments above.
review_title: null

Meta-review: metareview: **Strengths:**
•	The problem is relevant

•	The paper is well written with a clear presentation of its contributions.

•	Novel approach to pivoting task for in-hand manipulation.

**Weaknesses:**

•	Poor justification of the baseline model to compare the results of the proposed approach. 

•	MLP design is not well justified. 

•	Paper lacks many details in experimental sections.

•	Design choices are not well justified.

Post-rebuttal: 
Please revise the paper according to the points discussed in the rebuttal phase for camera-ready submission. 
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)