In-Hand Object Rotation via Rapid Motor Adaptation
Authors: Haozhi Qi
Ashish Kumar
Roberto Calandra
Yi Ma
Jitendra Malik
Venue: CoRL 2022 Poster (2022)
Keywords: Dexterous In-Hand Manipulation
Object Rotation
Reinforcement Learning

Abstract: Generalized in-hand manipulation has long been an unsolved challenge of robotics. As a small step towards this grand goal, we demonstrate how to design and learn a simple adaptive controller to achieve in-hand object rotation using only fingertips. The controller is trained entirely in simulation on only cylindrical objects, which then – without any fine-tuning – can be directly deployed to a real robot hand to rotate dozens of objects with diverse sizes, shapes, and weights over the z-axis. This is achieved via rapid online adaptation of the robot’s controller to the object properties using only proprioception history. Furthermore, natural and stable finger gaits automatically emerge from training the control policy via reinforcement learning. Code and more videos are available at https://github.com/HaozhiQi/Hora .

In-Hand Object Rotation via Rapid Motor Adaptation
Haozhi Qi
Ashish Kumar"
Roberto Calandra"
Yi Ma
Jitendra Malik 1.2
UC Berkeley
M Al
https://haozi: io/hora/
Train in Simulation
Directly Deploy in the Real World
14p
arXiv:2210.04885N [cs.RO] 10 Oct 2022
53)cm
200g
189
10 cm
6EA 7m
05S. 6.55
I5 K cm
22
21
105
13
Figure I: Left: Our controller is trained only in simulation simple cylindrical objects of different sizes and weights. Right: Without any real world fine-tuning, the controller can be deployed to real robot diverse of objects with different shapes, sizes and weights (object mass and the shortest/longest diameter axis length along the fingertips are shown the figure) using only proprioceptive information. Website: Emergence of natural stable finger gaits can be observed in the leamned control policy.
Abstract: Generalized in-hand manipulation has long been an unsolved challenge of robotics. As small step towards this grand goal, we demonstrate how to design and learn simple adaptive controller to achieve in-hand object rotation using only fingertips. The controller is trained entirely in simulation on only cylindrical objects, which then without any fine-tuning can be directly deployed robot hand to rotate dozens of objects with diverse sizes, shapes, and weights over the z-axis. This is achieved via rapid online adaptation of the robot's controller to the object properties using only proprioception history. Furthermore, natural and stable finger gaits automatically emerge from training the control policy via reinforcement learning. Code and more videos are available at our Website.
real
Keywords: In-Hand Manipulation, Object Rotation, Reinforcement Learning
1 Introduction
Humans are remarkably good at manipulating objects in-hand they can even adapt to new objects of different shapes, sizes, mass and materials with apparent effort. While several works have shown in-hand object rotation with real-world multi- fingered hands for single or a few objects [1, 2, 3, 4], truly generalizable in-hand manipulation remains unsolved challenge of robotics.
In this paper, we demonstrate that it is possible to train an adaptive controller capable of rotating diverse objects over the z-axis with the fingertips of multi-fingered robot hand (Figure 1). This task a simplification of the general in in-hand reorientation task, yet still quite challenging for robots since all times the fingers need to maintain dynamic static force closure the object to prevent it from falling (as it can not make use of any other supporting surface such as the palm).
Equual Contribution.
6th Conference on Robot Learning (CoRI. 2022), Auckland, New Zealand.

---

Our approach is inspired by the recent advances in legged locomotion [5, 6] using reinforcement learning. The core of these works is to learn compressed representation of different terrain properties (called extrinsics) for walking, which is jointly trained with the control policy. During deployment, the extrinsics is estimated online and the controller can perform rapid adaptation to it. Our key insight is that, despite the diversity of real-world objects, for the task of in-hand object rotation, the important physical properties such as local shape, mass, and size as perceived by the fingerips can be compressed to compact representation. Once such compressed representation (extrinsics) of different objects is learned, the controller can estimate it online from proprioception history and use it to adaptively manipulate diverse set of objects.
Specifically, we encode the object's intrinsic properties (such as mass and size) to an extrinsics vector, and train an adaptive policy with it as input. The learned policy can robustly and efficiently rotate different objects in simulation environments. However, we do not have access to the extrinsics when deploy the policy in the real world. To tackle this problem, to learn an adaptation module which estimate the extrinsic vector, using the discrepancy between observed proprioception history and the commanded actions. This adaptation module can also be trained solely in simulation via supervised learning. The concept of estimating physical properties using proprioceptive history has been widely used in locomotion [5, 6, 7] but has not yet been explored for in-hand manipulation.
use the rapid motor adaptation [6]
Experimental results a multi-finger Allegro Hand [8] show that our method can successfully rotate over 30 objects of diverse shapes, sizes (from 4.5 cm to 7.5 cm), mass (from 5g to 200 g), and other physical properties (e.g., deformable or soft objects) in the real world. We also observe that an adaptive and smooth finger gait emerges from the learning process. Our approach shows the surprising effectiveness of using only proprioception sensory signals for adaptation different objects, even without the usage of vision and tactile sensing. To further understand the underlying mechanisms of our approach, we studied the estimated extrinsics when manipulating different objects. We find interpretable extrinsic values that correlate to mass and scale changes, and that a low -ww-imensional structure of the embedding does exist, both of which are critical to our generalization ability.
2 Related Work
Classic Control for In-Hand Manipulation. Dexterous in-hand manipulation has been an active research area for decades [9]. Classic control methods usually need an analytical model of the object and robot geometry to perform motion planning for object manipulation. For example, [10, 11] rely on such a model to plan finger movement to rotate objects. [12] assumes objects are piece-wise smooth and use finger-tracking to rotate objects. [13, 14] demonstrate reorientation of different objects in simulation by generating trajectories using optimization. There have been also attempts to deploy systems in the real-world. For example, [15] calculates precise contact locations to plan sequence of contact locations for twirling objects [16] plans over a set of predefined grasp strategies to achieve object reorientation using two multi-fingert hands. [17, 18] use throwing external forces to perturb the object in the air and re-grasp it. Recently, works such as [19, 20] do in in-grasp manipulation without breaking the contact. [4] demonstrates complex object reorientation skills using a non-anthropomorphii hand by leveraging the compliance and an accurate pose tracker. The diversity of the objects they manipulate is still limited due to the intrinsic complexity of the physical world. In contrast to traditional control approaches which may use heuristics or simplified models to solve this task, we instead use model-free reinforcement learning train an adaptive policy, and use adaptation to achieve generalization.
Reinforcement Learning for In-Hand Manipulation. To get around the need of an accurate object model and physical property measures, in the last few years, there has been a growing interest in using reinforcement learning directly in the real-world for dexterous in-hand manipulation. [21] learns simple in- g-aaspp rolling for cylindrical objects. [22, 23] learns dynamics model and plan over it for rotating objects on the palm. [24, 25] use human demonstration accelerate the learning process. However, since reinforcement learning is very sample inefficient, the learned skills are rather simple or have limited object diversity. Although complex skills such as re-orientating a diverse set of objects [26, 27, 28] and tool use [29, 30) can be obtained in simulation, transferring the results to real-world remains challenging. Instead of directly training a policy in the real world, our approach learns the policy entirely in the simulator and aims to directly transfer to the real world.
Sim-to-Rea Transfer via Domain Randomization. Several works aim to train reinforcement learning policies using a simulator and directly deploy it in a real-world system. Domain randomiza-

---

Trainable Module in Red
Base Policy Training
Oc
Base Pollccy (T)
object position, scale, center of coefficient of friction
Object Prup Encoder (1)
Ze
Copy and Froeze
1| 2x 112
Qt-29, ag-30
Adaptation Module (+)
2
Base Policy (11)
9t. Qt-1
at
Adaptation Module Training
Copy and Freeze
Copy Freeze
Qr-29, Qg-30
Adaptation Module (+)
2e
Base Policy (#1)
20
Ac-1
at
Deployment
Figure 2: An overview of our approach at different training and deployment stages. In Base Policy Learning, we jointly optimize yi and æ using PPO [33]. The observation or only contains three past joint positions and commanded actions. Next, in Adaptation Module Learning, we freeze the policy and supervised learning to train é which proprioception and action history estimate the extrinsics vector zN. During Deployment, the base policy uses the extrinsics ε, estimated and updated online by d.
tion [31] varies the simulation parameters during training expose the policy diverse simulation environments so that they can be robustly deployed in the real-world. Representative examples are [1] and [2]. They leverage massive computational resources and large-scale reinforcement learning methods to learn an agile object reorientation skills and solve Rubik's 's Cube with single robot hand. However, they still focus only on manipulating limited number of objects. [3] learns finger-gaiting behavior efficiently and transfers to a real robot when hand facing downwards, but they do not only use fingertips and the objects considered diverse set of objects and can be trained within a few hours.
all cubes. Our approach focuses on generalization
Sim-to-Real via Adaptation. Instead of relying on domain randomization which agnostic to current environment parameters, [32] performs system identification via initial calibration, or online adaptive control estimate the system parameters for Sim-to-Ree Transfer. However, learning the exact physical values and alignment between simulation and real-world may be sub-optimal because of the intrinsic inaccuracy of physics simulations. An alternative way is to learn low dimensional embedding which encodes the environment parameters [5, 6] which is then used by the control policy to act. This paradigm has enabled robust and adaptive locomotion policies. However, not straight straightforward to apply it on the in-hand manipulation task. Our approach demonstrates how to design the reward and training environment to enable a natural and stable controller that can transfer to the real world.
3 Rapid Motor Adaptation for In-Hand Object Rotation
An overview of our approach is shown in Figure 2. During deployment (Figure 2, bottom), policy infers a low-dimensional embedding of object's 's properties such as size and mass from proprioception and action history, which is then used by our base policy rotate the object. We first describe how we train a base policy with object property provided by a simulator, then we discuss how to train an adaptation module that is capable of inferring these properties.
3.1 Base Policy Training
Privileged Information. In this paper, privileged information refers to the object's properties such as position, size, mass, coefficient of friction, center mass of the object. This information, denoted by 9-dim vector R at timestep t, can be accurately measured in simulation. We provide

---

this as an input to the policy, but instead of using directly, we use an 8-dim embedding (called extrinsics in [6]) 2, = μle,) which gives better generalization behavior as we show in Section 5.
Base Policy. Our control policy takes the current robot joint positions qe E R 16 the predicted action - E R at the last timestep, together with the extrinsics vector x, E R as input, and outputs the target of the PD Controller (denoted as at). We also augment the observation to include two additional timesteps to have the velocity and acceleration information. Formally, the base policy output ar = π T zx) where Og (qt-2:t 2:t; Ot 3:t E TR
Reward Function. We jointly optimize the policy л and the embedding μ using PPO [33]. The reward function depends on several quantities: w is the object's angular velocity. & is the desired rotation axis (we use the z-axis in the hand coordinate system). quit is the starting robot configuration. is the commanded torques at each timestep. is the object's 's linear velocity. Our reward function (subscript omitted for simplicity) to maximize then
r = Trot + Apose ppss + Alinvel Plinvel + Awork Twork + Atorque Itorque
(1)
where Frot = max(min(w k Tmux)) Tmin) is the rotation reward, Tpose = - 11q quiit 2 is the hand pose deviation penalty, Ytorque = - || is the torque penalty, Twork = --" q is the energy consumption penalty, and Fliuvel - - ||0l is the object linear velocity penalty. Note that in contrast to [28] which explicitly encourages at least three fingertips to always be in contact with the object, we do not enforce any heuristic finger gaiting behaviour. Instead, a stable finger gaiting behaviour emerges from the energy constraints and the penalty on deviation from the initial pose.
Object Initialization and Dy Dynamics Randomization. A good training environment has to provide enough variety in simulation to enable generalization in the real world. In this work we find that using cylinders with different aspect ratios and masses provides such variety. We uniformly sample different diameters and side lengths of the cylinder.
We initialize the object and the fingers in a stable precision grasp. Instead of constructing the fingertip positions as in [28], we simply randomly sample the object position, pose, and robot joint position around a canonical grasp until a stable grasp is achieved. We also randomize the mass, center of mass, and friction of these objects (see appendix for the details).
3.2 Adaptation Module Training
We cannot directly deploy the learned policy и to the real world because we do not directly observe the vector e, and hence cannot compute the extrinsics ž₁. Instead, we estimate the extrinsics vector & from the discrepancy between the proprioception history and the commanded action history via an adaptation module é. This idea is inspired by recent work in locomotion [5, 6] where the proprioception history is used to estimate the terrain properties. We show that this information can also be used to estimate the object properties.
To train this network, we first collect trajectories and privileged information by executing the policy a, = T (o 2,) with the predicted extrinsic vectors &, = ((q:-k:t> - Mcanwhile we also store the ground-truit -tuth extrinsic vector and construct training set
B {(q {(q 1 z (9) a.x"( ) )-
(8) (1)
Then we optimize & by minimizing the Eg distance between 2, and &, using Adam [34]. The process is iterative until the loss converges. We apply the same object initialization and dynamics randomization setting as the above section.
4 Experimental Setup and Implementation Details
Hardware Setup. We use an Allegro Hand from Wonik Robotics [8]. The Allegro hand is a dexterous anthropomorphic robot hand with four fingers, with each finger having four degrees of freedom. These 16 joints are controlled using position control at 20 Hz. The target position commands are converted to torque using a PD Controller (K, = 3.0, K2 = 0.1) at 300 Hz.
Simulation Setup. We use the IsaaeGym [35] simulator. During training, we use 16384 parallel environments to collection samples for training the agent. Each environment contains a simulated Allegro Hand and an cylindrical object with different shape and physical properties (the exact parameters are the supplementary material). The simulation frequency is 120 Hz. and the control frequency is 20 Hz. Each episode lasts for 400 control steps (equivalent to 20 s).

---

Within Training Distribution
Out-of-Distriuttin
Method
RotR (1) TTF (1) ObjVel (4)
Torque (L)
RotR (t)
TTF (1) ObjVel (C)
Torque ())
Expert
233
0.85
0.28
1.24
165.07
0.71
0.422n06
1.24
Periodic
43.62
0.44
0.72
22.45 2o:0
0.34
1.1 1 12
1.4 inss
NoAdapt
90.89-455
+007
0.44
a.34
54.50=331
0,51 #0.06
0.63
1.34
DR
12
0.81 zo02
g.34
1,42
140.8
0.63
0.64
1.48
SysID
174,42
0.81 1002
0.32
1.29
132.56
0.62
0.50
1.26
Ours
2
0.82
0.29
1.20
160.60
0.68
0.47
1.20
Table
rion; 2) compare our method to several baselines in simulation two settings: 1) Within Training Distribu- Our method with online continuously adaptation achieves the best performance compared to all the baselines, closely emulating the performance of the Expert that has the privileged information the input.
Baselines. We compare our method to the baselines listed below. We also compare with the policy with access to privileged information (Expert), as the upper bound of our method (Figure 2, top row). 1. A Robust Policy trained with Domain Rundomization (DR): This baseline is trained with the same reward function but without privileged information. This gives a policy which is robust, instead of udaptive, to all the shape and physical property variations [1, 3, 2, 36].
2. Online Explicit System Identification (SysID): This baseline predicts the exact system parameters
e instead of the extrinsic vector 2, during training the adaptation module.
3. No Online Adaptation (NoAdapt): During deployment, the extrinsics vector 2; is estimated at the first time step and stays frozen during the rest of the run. This is to study the importance of online adaptation enabled by the adaptation module ø.
4. Action Replay (Periodic): We record a reference trajectory from the expert policy with privi- leged information and run it blindly. This is to show our policy adapts to different objects and disturbances instead of periodically executing the same action sequence.
Metries, We use the following metrics to compare the performance of our method to baselines.
Time-to-Fall (TTF). The average length of the episode before the object falls out of the hand. This value is normalized by the maximum episode length (20s in simulation experiments and 30s in the real world experiments).
2. Rotation Reward (RotR). This is the average rotation reward (w k)) of an episode in simulation. Note that we do not train with this reward. Instead, we use clipped version of this reward during training.
3. Radians Rotated within Episode (Rotations). Since angular velocity of the object is hard to accurately measure the real world, we instead measure the net rotation of the object (in radians) achieved by the policy with respect to the world's z-axis. This metric is only used in the real world experiments.
4. Object's 's Linear Velocity (ObiVel). We measure the magnitude of the linear velocity to measure the stability of the object. This is only measured simulation. The value is scaled by 100.
5. Torque Penalty (Torque). We measure the average & norm of commanded torque per timestep during execution to measure the energy efficiency.
5 Results and Analysis
In this section, we compare the performance of our method to several baselines both in simulation and in real-world deployment. We also analyze what the adaptation module learns and how changes during policy execution and as the objects change. Finally, we focus on training policy for rotating an object along the negative z-axis with respect to the world coordinate. We also explore the possibility of training multi-axis policy (#t z-axis) in the appendix. More qualitative results of our method and several ablations of our method are in our Project Website and the supplementary.
5.1 Generalization via Adaptation
Comparison in Simulation. We first compare our method with the baselines mentioned Section in simulation. We evaluate all methods under two settings: 1) In the Within Training Distribution
5

---

Method
Rotations (1)
TTF (1)
Torque (4)
165 g
139 g
106
200 g
181
DR
9.37
0.72
2.03
SysID
10.36
0.61 +0.3
1.88
NoAdapt
N.A.
0.35
N.A.
Ours
23.96
0.98
1.84
Figure 3: Quantitative Evaluation on a diverse set of Heavy Objects (Left). Our method which uses adaptation performs the best in terms of total rotated angle (in radians). Time To Fall (TTF), and Energy efficiency (Torque). The DR baseline has conservative policy which results in slower angular velocity. SysID has more dynamic and agile policy but very unstable as be seen in lower TTF compared to DR and ours. The NoAdapt baseline fails on the task showing the importance continuous online adaptation.
setting, we use the same object set and randomization setting as in RL training; 2) In the Out-of- Distribution setting, we use objects with larger range of physical randomization range. We also change 20% of the objects to be spheres and cubes. We compute the average performance over 500K episodes with different parameter randomizations and initialization conditions. We report the average and standard deviation over five models trained with different seeds.
The results in Table 1 show that our method with online adaptation achieves the best performance compared to all the baselines. We see that adaptation to shape and dynamics of the object not only enables a better performance in training. but also gives a much better generalization out- of-distribution object parameters compared to all the baseline methods. The Periodic baseline (i.e., simply playback of the expert policy) does not give a reasonable performance. Although it can rotate the exact same object with the same initial grasp and dynamic parameters, it fails to generalize beyond this very narrow setup. This baseline helps us understand the difficulty of the problem. The NoAdapt also performs poorly compared to our method which uses continuous online adaptation. The weaker performance of this baseline is explained by the fact that it does not update the extrinsics during the episode. This shows the importance of continuous online adaptation. The DR baseline, although roughly matches or performs better than the other baselines in terms of RotR and TTF, it worse in other metrics related to object stability and energy efficiency. This is because the DR baseline is unaware of the underlying object properties and needs to learn a single gait, instead of an adaptive one, for all possible objects. The SyxID baseline also performs worse than our method in both of the evaluation distributions. This is because it is harder as well as unnecessary to learn the exact values of the shape and dynamics parameters to adapt. This comparison shows the benefit of learning -ondiimnnsional compact representation which has a coarse relative activation for different physical properties instead of the exact value (see Figure and Figure 6).
Comparisons. Next, we show real world comparisons of our policy to the baselines on two set of challenging objects shown in Figure 3 and Figure 4. We exclude the Periodic baseline because it does not work well even in Simulation and Expert because we cannot access privileged information in the real world. We evaluate each method using 20 different initial grasping positions and 6 different objects in each set. The maximum episode length is s0 We first study the behavior of policy and different baselines in set of heavy objects (more than 100 g) including a baseball, different fruits, vegetables, and a cup (Figure 3). Our method performs significantly better in all metries. Our method can achieves consistent rotation for almost all trials without falling down with an average Rotations of 23.96 (with a equivalent rotation velocity 0.8 rads-)) We find the DR baseline is very slow and conservative in all the trials since it is unaware of object properties and needs to learn a single gaiting behaviour for all objects. As a result, it has the lowest Rotations, although the TTF is higher than the SysID baseline. We also find the DR could perform reasonably well on objects of larger sizes because they are easier to rotate and forms a reasonable chunk of training distribution. The SysID baseline learns a more dynamic and agile behavior. having better Rotations metric, but lower TTF and Rotations showing that precise estimation of system parameters both unnecessary and difficult. We find that it is particularly hard for it to generalize to the small cup and the slightly soft tomato. Lastly, we observe similar behavior as in [6] that the NoAdapt baseline does not transfer successfully to the real world, showing the importance of continuous online adaptation for successful real-world deployment.
We perform the same comparisons on a collection of irregular objects (Figure 4). It contains a container with moving COM, objects with concavity, cylindrical kiwi fruit, shuttlecock, toy with

---

Method
Rotations (1)
TTF (1)
Torque (1)
High COM
Concave
Cylinder
Squrr
DR
6.59
0.66
1.85
SysID
8. 8.16
0.46
1.70a0
NoAdapt
N.A.
0.12
N.A.
Ours
19.22
0.78
1.48
Figure 4: Quantitative Evaluation on a diverse set of Irregular Objects (Left). Our method can successfully generalize to rotating diverse set of objects including objects with holes, soft and deformable objects (none of these were included in the training). Our method outperforms the baselines on all the metrics. The DR baseline has the second highest TTF but low Rotations because outputs very conservative and slow trajectories. SyslD achieves slightly faster but a very unstable policy. The superior performance of our method over baselines shows the importance of adaptation via a low dimensional extrinsics estimation for generalization in this task.
30
200
106
19-4, 5.4) cm
(53, 6:51 cat.
(PA) cm
6.00
0.00
0.5
Eaniii
4.80
4..5
0.45
Dontia Vahe
0.30
Time
Two of the 8-dim estimated extrinsics vector during one continuous run in which we change object in the hand every 30s for 6 objects. The top plot shows the extrinsic value 21,0 which responds to changes in object diameter with smaller diameters leading to lower values. Since the perceived diameter of the object changes during rotating irregular shaped object, object. The bottom plot shows the correlation between žt,2 and object mass: attains higher values lighter objects and lower for heavier objects. See this link for video.
Figure 5:
variations this extrinsic value within the same
holes, and cube toy. Rotating the cube is particularly difficult because we only rely on usage of fingertips. Although these variations are challenging for this task and are beyond what is seen during training, our method still performs reasonably well, outperforming all the baselines. We see that the DR baseline can perform stable but slow in-hand rotation for the container and the shuttlecock, but largely fails for the other objects, indicating its difficulty in shape generalization. For the Sys/D baseline, its stability is significantly lower than the DR baseline as well as our method, despite having higher angular velocity. The NoAdapt baseline performs similarly to what we observed in Figure 3.
5.2 Understanding and Analysis
To understand how our method generalizes to a diverse set of objects, we design few experiments and visualization to help develop some insights.
Extrinsics over Time. We run one continuous evaluation episode in the real world in which replace the object in the hand every 30s for 6 objects. Note that during training, never randomize the objects within an episode. During the entire run, we record the estimated extrinsics and plot out of the 8-dim extrinsics vector in Figure 5. The top plot shows the extrinsic value 21,0 which responds to changes in object diameter. It has lower value for smaller diameters and higher value for larger diameters. The bottom plot shows that the extrinsic value 21,2 responds to variations in object mass.
Extrinsic Clustering. Another way to understand the estimated extrinsics á is by clustering extrinsics vector estimated while rotating different objects. In Figure 6, we visualize the estimated extrinsics vector using t-SNE for rotating different objects. We find that objects of different sizes and different weights tend to occupy separate regions explained in Figure 6.
7

---

IS 60
(55,6
260g
coka
13:3
D4
22
10:22
oflor
10:
-20
(SS c3ii
45g
cakor
coker
10
-20 0
60
Figure 6: We visualize the estimated extrinsics vector 2, using t-SNE for rotating 6 different objects for seconds each. We found that objects different sizes different weights tend to occupy separate regions. example, & corresponding to smaller objects tend to cluster on the hottom left. The object with ID (shown in the bottom left in the above figure) produces scattered 2 because of its irregular shape, which results in an equivalently varying scale of the object. This locality leads to the scattering of the extrinsics across the t-SNE plot, and this locality also what gives us generalization as evaluated in Table Figure 3 Figure
Emergent Finger Gaits. We find it important to use cylindrical objects for the emergence of stable and high-clearance gait. On website, compare the learned finger gaits when training with cylindrical objects and with pure spherical objects. The latter training scheme leads to policy with a dynamic gait which works well on balls but fail to generalize to more complex objects.
5.3 Real World Qualitative Results
On our Project Website, we qualitatively compare our method with the baselines on different objects and further evaluate our method on 30 objects of different shapes and physical properties, including porous objects and non-rigid objects. The diameter ranges from 4.5 cm to 7.5 em and the mass ranges from 5 g to 200 g. Note that only cylindrical objects of different sizes and aspect ratios during training and show shape generalization insight which allows this generalization is that the shape of the objects perceived by the fingertips of the hand can compressed to a low dimensional space. We estimate this low dimensional space from proprioceptive history, which allows to generalize to objects which are seemingly different in the real world but might look similar in the extrinsics space.
the real world with the use of rapid adaptation. The key
6 Discussion and Limitations
There different levels of difficulty for general dexterous in-hand manipulation. The task considered in this paper (in-hand object rotation over the z-axis) is a simplification of the general SO(3) reorientation problem. However, it is not limiting simplification. With three policies for rotation along three principle axes, we can achieve rotating the object to any target pose. We view this task as an important future extension of our work.
We show the feasibility of purely proprioceptive in-hand object rotation for diverse set of objects. We find most of our failure cases the real-world experiments are due incorrect contact points which leads to unstable force closure. Since our method only relies on the proprioceptive sensing, it unaware of the precise contact positions between the objects and the fingertips. Another failure case is when the object to be rotated is tiny (with diameter less than 4.0 cm). In this case the fingers will frequently collide with each other, making the robot not able to maintain the grasp balance of the object. More extreme and sophisticated shapes are also harder to manipulate. For those more challenging tasks, might be necessary incorporate tactile or visual feedback.
We aim to study the generalization to the real world via adaptation, so we do not utilize real-world experience to improve our policy. Incorporating real-world data to improve our policy (e.g. by using meta-learning) will be interesting and meaningful next step.

---

Acknowledgments
This research was supported as BAIR Open Research Common Project with Meta. In addition, in their academic roles at UC Berkeley, Haozhi, Ashish, and Jitendra were supported in part by DARPA Machine Common Sense (MCS) and Haozhi and Yi by ONR (N00014-20-1-2000 and N00014-22-1- 2102). We thank Tingfan Wu and Mike Lambeta for their generous help on the hardware setup, Xinru Yang for her help on recording real-world videos. We also thank Mike Lambeta, Yu Sun, Tingfan Wu, Huazhe Xu, and Xinru Yang for providing feedback on earlier versions of this project.
References
[1] OpenAI, M. Andrychowicz, B. Baker, M. Chociej, R. Józefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, J. Schneider, S. Sidor, J. Tobin, P. Welinder, L. Weng, and W. Zaremba. Learning Dexterous In Hand Manipulation. The International Journal of Robotics Research (IJRR), 2019. doi: 10.1:0.11100 1000000177/0277844999888887
[2] OpenAI, I. Akkaya, M. Andrychowicz, M. Chociej. M. Litwin, B. McGrew, A. Petron, Paino, M. Plappert, G. Powell, R. Ribas, J. Schneider, N. Tezak, J. Tworek, P. Welinder, L. Weng, Q. Yuan, W. Zaremba, and L. Zhang. Solving Rubik's Cube with Robot Hand. arXiv preprint arXiv: 19t0009900.33 11,,0711,,,,0 2019. doi: 1..48550/arXiv walaaaXiiiv.11...000 1910.07113.
[3] L. Sievers, J. Pitz, and B. Bäuml. Learning Purely Tactile In In-Hand Manipulation with Torque-Contn Controlled Hand. International Conference on Robotics and Automation (ICRA), 2022. doi: 200.4448550/XX...22...000
[4] A. Morgan, K. Hang, B. Wen, K. E. Bekris, and A. Dollar. Complex In-Hand Manipulation via Compliance-Fnablle Enabled Finger Gaiting and Multi- Modai Planning. IEEE Robotics and Automation Letters (RA-L.), 2022. doi: 109/LO0..00000.000
[5] Lee, Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter. Learning Quadrupedal Locomo- tion over Challenging Terrain. Science Robotics, 2020. doi: 10. 12//scirooooticcsace55666
[6] A. Kumar, Z. Fu, D. Pathak, and J. Malik. RMA: Rapid Motor Adaptation for Legged Robots.
In Robotics: Science and Systems (RSS), 2021. doi: 10.15607/RSS..000 10111 XVVI1011.
[7] Z. Fu, Kumar, A. Agarwal, H. Qi, J. Malik, and D. Pathak. Coupling Vision and Propriacep- tion for Navigation of Legged Robots. In Computer Vision and Pattern Recognition (CVPR), 2022. 1oi:10..8550/arra 400
[8] WonikRobotics. AllegroHand.
hwtp:/ottosscccoo...oooooo.o 2013.
[9] A. M. Okamura, N. Smaby, and M. R. Cutkosky. An Overview of Dexterous Manipulation. In International Conference on Robotics and Automation (ICRA), 2000. doi:10. 100//OBOT. 8000.844067.
[10] I. Han and J. C. Trinkle. Dextrous Manipulation by Rolling and Finger Gaiting. In International Conference on Robotics and Automation (ICRA), 1998. 10 100.1.11////B00.199900 667066.
[11] J.PP. Saut, A. Sahbani, S. El-Khoury, and V. Perdereau. Dexterous Manipulation Planning Using Probabilistic Roadmaps in Continuous Grasp Subspaces. In International Conference on Intelligent Robots and Systems (IROS), 2007. doi:
[12] D. Rus. In-Hand Dexterous Manipulation Piecewise-Smootth 3-D Objects. The International Journal of Robotics Research (IJRR), 1999. do:100.177002773349992666
[13] Y. Bai and C. K. Liu. Dexterous Manipulation Using Both Palm and Fingers. In International Conference Robotics and Automation (ICRA), 2014. doi: 10::10.11000 100.1100/CCRA22144699770
[14] I. Mordatch, 7. Popović, and F. Todorov. Contact-Invariant Optimization for Hand Manipulation. In Eurographics, 2012. doi: (/233///CC//CA/1//1133
[15] R. Fearing. Implementing Force Strategy for Object Re-orientation. In International Confer- ence on Robotics and Automation (ICRA), 1986. 10000000.199

---

[16] R. Platt, A. H. Fagg. and R. A. Grupen. Manipulation Gaits: Sequences of Grasp Control Tasks. In International Conference on Roboties and Automation (ICRA), 2004. doi: 100//OOOO. 2004.1307247.
[17] N. Furukawa, A. Namiki, S. Taku, and M. Ishikawa. Dynamic Regrasping Using a High-speed Multifingered Hand and High-speed Vision System. In International Conference on Robotics and Automation (ICRA), 2006. doi: 1001000000.0.2000.16600 1641181.
[18) N. C. Dafle, A. Rodriguez, R. Paolini, B. Tang, S. S. Srinivasa, M. Erdmann, M. T. Mason, I. Lundberg, H. Staab, and T. Fuhlbrigge. Extrinsic Dexterity: In-Hand Manipulation with External Forces. In International Conference on Robotics and Automation (ICRA), 2014. 1tt:00.11//CRR..200 66900062.
[19] C. Teeple, B. Aktas, M. C.-S. Yuen, G. Kim, R. D. Howe, and R. Wood. Controlling Palm- Object Interactions Via Friction for Enhanced In-Hand Manipulation. IEEE Robotics and Automation Letters (RA-L), 2022. doi: 100.1009/RR.22222.14.4.....
[20] B. Sundaralingam and T. Hermans. Golaxed-Rigidij Constraints: Kinematic Trajectory Op- timization and Collision Avoidance for In-Grasp Manipulation. Autonomous Robots, 2019. doi: 1007/s.0 4-972-77
[21] H. Van Hoof. T. Hermans, G. Neumann, and J. Peters. Learning Robot In-Hand Manipulation with Tactile Features. In International Conference on Humanoid Robots (Humanoids), 2015. doi:10. Noi:10.1109/HUNI /O//HU/ANOIISSS..O W01..OI0.S.......3....
[22] V. Kumar, F. Todorov, and S. Levine. Optimal Control with Learned Local Models: Application to Dexterous Manipulation. In International Conference on Robotics and Automation (ICRA),
2016. doi: 109/ 6.7881156.
[23] A. Nagabandi, K. Konolige, S. Levine, and V. Kumar. Deep Dynamics Models for Learning Dexterous Manipulation. In Conference Robot Learning (CoRL), 2019. 10i:50..850//rxiiv.
11652.
[24] A. Gupta, C. Eppner, S. Levine, and P. Abbeel. Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstration. In International Conference on Intelligent Robots and Systems (IROS), 2016. 10 20W1//RRO..20..77559957
[25] V. Kumar, A. Gupta, E. Todorov, and S. Levine. Learning Dexterous Manipulation Policies from Experience and Imitation. arXiv, 2016. 1oi:10048855/arxii 1611.05095.
[26] W. Huang, I. Mordatch, P. Abbeel, and D. Pathak. Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning. arXiv, 2021. doi: 0000100045500/XX21111..6666
[27] T. Chen, J. Xu, and P. Agrawal. A System for General In-Hand Object Re-Orientation. In Conference on Robot Learning (CoRL), 2022. doi: 10.. 11..0004..
[28] G. Khandate, M. Haas-Heger, and M. Ciocarlie, On the Feasibility of Learning Singer-gaiting In-hand Manipulation with Intrinsic Sensing. In International Conference on Robotics and Automation (ICRA), 2022. doi: 10.1 (ttpicaa446692222229811111
[29] A. Rajeswaran, V. Kumar, A. Gupta, G. Vezzani, J. Schulman, Todorov, and S. Levine. Learn- ing Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations. In Robotics: Science and Systems (RSS), 2018. doi:10.11607/RSS. 2018.XIV.049.
[30] I. Radosavovic, X. Wang, L. Pinto, and J. Malik. State-Only Imitation Learning for Dexterous Manipulation. In International Conference on Intelligent Robots and Systems (IROS), 2021. 10::0..109/RO05/11 68. 2021.9636557.
[31] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel. Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World. In International conference on intelligent robots and systems (IROS), 2017. doi: 1ef:10.11999RA 109/ROOS.2017 23..02213......1133....3.3.....3
[32] Y. Karayiannidis, C. Smith, D. Kragic, et al. Adaptive Control for Pivoting with Visual and Tactile Feedback. In International Conference on Robotics and Automation (ICRA), 2016. doi: 100.1110//RA..2.1......
10

---

[33] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal Policy Optimization Algorithms. arXiv, 2017. 10011004455///xx..1100000 1707.063477
[34] D. P. Kingma and Ba. Adam: A Method for Stochastic Optimization. International Conference Learning Representations (ICLR), 2015. 40a.1000////r
[35] V. Makoviychuk, L. Wawrzyniak, Y. Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin,
A. Allshire, A. Handa, and G. State. Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning. arXiv, 2021. d6::10:485500arxx2210...864
[36] A. Allshire, M. Mittal, V. Lodaya, V. Makoviychuk, D. Makoviichuk, F. Widmaier, M. Wüthrich,
S. Bauer, A. Handa, and A. Garg. Transferring Dexterous Manipulation from GPU Simulation Remote Real- Real-World Trifinger. arXiv, 2021. doi: 1e.100.488 10.44485550002222218..777777
[37] D.-A. Clevert, Unterthiner, and Hochreiter. Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUss. In International Conference on Learning Representations (ICLR), 2016. 10i:10.48550/am 15111071897


Reviews:
[{"review_id": "t6jLjyAsXmE", "reviewer": "Reviewer_arnN", "summary": "The paper employs Rapid Motor Adaptation to tackle the problem of policy generalization in in-hand manipulation tasks. The paper mainly describes results from a new experimental setup of in-hand manipulation. Generalization technique is along the similar lines of RMA, which tries to extract extrinsics vector from proprioceptive history of interaction with the world (environment). This extrinsics vector is hence used by a policy network to adapt its behavior to sensed changes in the environment. ", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n- The paper sufficiently motivates the problem statement.  \n- The paper presents an interesting application of an adaptation technique previously demonstrated for a different experimental setup. \n- The qualitative hardware results seem sufficiently convincing and validate the applicability of the underlying method.\n- The quantitive results (Figure 3) aptly explain how different methods respond to environmental perturbations. \n- The presented experiments cover impressive variations in object shapes and dimensions. \n- The ablation study is thorough and provides sufficient details about the training and testing procedures. \n\n### Weaknesses\n- Most importantly, the paper only presents a new application of an existing idea and lacks any new ideas. \n- Furthermore, it does not present new insights on the existing idea (for instance, why having a separate extrinsic vector prediction network is a good design choice over directly training a meta-learning based policy with adaptive behavior).  \n- The paper and the underlying method’s adaptive functioning is limited by the ability to simulate environment/robot variations. It would be interesting if authors could provide any insights on method’s response to variations not previously seen during training. (For instance, by freezing one of the known variable during training and seeing how the method responds during testing.)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper presents a new and interesting experimental setup of an exisiting method but lacks originality in ideas or providing deep insights. ", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {"issues": "- Although the paper demonstrated a novel use case of an existing method, it doesn't present any new idea or new insights from an existing idea which is not ideal. It would be interesting if it could additionally comment on new insights gained from running the underlying method for a new experimental setup. \n- It is not apparent to me how the policy behavior changes in response to a different environment (i.e., environmental variation). It would be helpful if this can be explicitly demonstrated as a video / plot. As an example, it would be interesting (at least to me) to see how the policy behavior changes with and without continuously using the adaptation module (e.g., by latching its output / using the same output as obtained from a different object).  \n- Could you also add qualitative results from the DR, SysID, NoAdapt, and Periodic baselines? In case I missed it, please point me to the correct videos. \n- Lines (42-43): What type of gaits emerged from the baselines? Could you talk more about why having an adaption module could have led to smoother gaits?\n- Lines (103-104): Could you explicitly states these changes?\n- It would be greatly appreciated if you could expand on the limitations section. \n\nI am looking forward to the discussion!\n\n### Minor issues\n- Lines (7-9): For a reader unaware of RMA, it can be slightly misleading what is being estimated by the method. \n- Line (36): It has some grammatical issues. (Please also look at the remaining introduction section as it has additional grammatical errors.)\n- Line (37): It is not clear to me what `adaptively estimate` means. Does it imply continuous updates?\n- Line (56, 57): I am not sure, but: Starting with citation number might be aberrant with the template. You might want to double check."}}, {"review_id": "cty_h5oSS_", "reviewer": "Reviewer_p2uD", "summary": "The paper presents a system that learns an adaptive controller for in-hand manipulation, trained in simulation and deployed on a real robotic hand. The proposed approach relies on reinforcement learning in simulation, and does not require retraining or fine-tuning on the real robot, thanks to an adaptation module. The proposed system also relied on the fact that objects' characteristics, such as their shape and mass, as perceived by the fingertips proprioception can be compressed into a low dimensional embedding. Such representation then allows generalisation to unseen objects.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper presents a solid execution of an important topic in robot manipulation, that is in-hand manipulation. In particular, the proposed solution generalises across objects and is shown to be reliable on a real robotic hand. The proposed solution is simple and clean, although it has limitations (mentioned by the authors as well), by being only relying on proprioception.\nThe paper is written and structure in a clear way, and it is pleasant to read. Figures, diagrams and videos are well made and help clarify the contributions. \nThe paper includes a clear description of the experimental setup, and implementation details are reported (also in the supplementary material submitted).\nExperiments include a comprehensive set of ablations and comparisons, which make the contribution of the paper more solid.\n\nWhat is the (quantitative) effect of the gap between simulation and real setups? While usually more relevant for visual-related features, the domain gap for proprioceptive measurements could be limited. Is this true in your case? what is the main discrepancy playing a role in your experiment? As your perception stack is not based on visual clues, the appearance of the object in simulation should not affect your transfer, however mass or COM could be more relevant in your setup. Which object characteristic posed the biggest challenge for the sim-to-real transfer using your method?\nThe reward definition proposed includes several terms that are also commonly used to regularise actions in RL: do you have ablation studies or empirical results to highlight which component is mostly affecting the final behaviour?\nCan you motivate the choice of using 2 time steps for your $o_t= (x_{t−2:t}, a_{t−3:t−1})$ in your policy?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper presents a solid design, implementation and execution of a method to address in-hand manipulation on a real robot hand with a variety of different objects. Although the components of the proposed framework are individually not novel on their own, their combination and the results presented are significant and the analysis conducted is thorough. ", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {"issues": "Discussion on the simulation-reality gap in the context of the presented experimental setup: which part of your perception and control stack are less robust?\nReward formulation: which terms have the most impact?\nPolicy: motivate choices.\n\nMinor: typos p.6 \"The NoAdapt also *has performs* poorly...\", \" Our method *can achieves* consistent\" "}}, {"review_id": "4yDNiT_5Usb", "reviewer": "Reviewer_5UyE", "summary": "This paper presents an algorithm to achieve in-hand object rotation through finger-gaiting using linkage-based robot hand. In the base policy training phase, a base policy was trained jointly with an object property encoder using proximal policy optimization with a custom-defined reward function. The object property encoder takes privileged object information and outputs an extrinsics vector, which, combined with the current robot joint position and the previous predicted robot action, are then used as the input of the base policy to output the next robot action. The next phase trained an adaptation module to output an estimated extrinsics vector using the proprioception history as input, in order to lift the dependency to privileged object information during deployment. The algorithm was tested in real world experiments using an Allegro hand, in comparison with different baseslines.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:** \n* Very clear presentation, especially section 3 with the help of Fig. 2.\n* Multiple real world experiments demonstrating the effectiveness of the proposed method.\n\n**Weaknesses:**\n* The demonstrated object rotation is 2D. This would have been fine if it was well addressed in the paper (either in the scope or in the limitation). \n* Because the manipulation is 2D, some of the experimental objects do not really demonstrate generalizability, because it is the 2D cross section that matters. Objects of different shapes could have very close cross sections. \n* Not sufficiently encode object geometry: there is no object geometry information encoded in the object property encoder because it was not used as input for training. This could also explain the fairly limited generalizability for 2D manipulation. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The authors are solving the general object reorientation problem. While the proposed method aims to show general 3D reorientation that can be generalized to different objects, the experiment results only demonstrated object reorientation around two opposing axes (essentially 2D reorientation). This also made the object generalization less impressive, because objects such as the badminton and the cup have similar 2D cross section as a sphere. However, the method is still valid and was executed very well even with the limited scope.  ", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are not well addressed", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {"issues": "* The outcome shows a potentially narrower scope compared to what was claimed in the paper. Based on the videos from the project page, the proposed method only achieved 2D manipulation: even though the objects are 3D, the rotations are all around the vertical axis ([0, 0, 1] and [0, 0, -1]). It is recommended to either claim the correct scope of the problem or provide further demonstrations to support the claim of general reorientation. \n* The effective 2D aspect ratio of all objects are very close to 1.  Although there are a few long objects present in the experiment, their cross sections (where the fingers are in contact) have aspect ratio close to 1. This is okay as long as properly addressed in the limitation section. What would happen if the objects being manipulated (or their cross sections where the fingers are in contact with) have more extreme aspect ratios?  What is limiting the object generalizability? It will be very interesting to see the extrinsics vector of an more extreme shape compared to the nice shapes. \n"}}, {"review_id": "2Y51cOQ69wo", "reviewer": "Reviewer_W5ch", "summary": "This paper applies the training pipeline of Rapid Motor Adaptation to the setting of in-hand object rotation. Note that in this paper, the `in-hand object rotation` actually means the robotic hand keeps rotating the objects along the fixed $z$-axis, not in the full $SO(3)$ space. The results show that the allegro hand can quasi-statically rotate around 30 different objects. The real-world experiments look nice.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strenghts:\n\n* The overall learning pipeline is simple and convincing.\n* The real-world experiments are nice and show that the policies actually transfer.\n* It's nice to see that policies trained with only rotating a cylinder can rotate other objects to some extent.\n\n\nWeaknesses:\n\n* The paper is a direct application of a published method to a new task. The method itself is unchanged from the prior works. \n* One baseline that is missing is the ablation of the adaptation module. While the paper has a baseline called NoAdapt, I think there is another baseline missing. The aspect that's missing is whether we actually need a separation of an adaptation module and a policy module. The paper currently regresses the output of the adaptation module to the privileged state embedding. Is this necessary? In other words, can we just train a temporal network $\\pi(x_{t-30}, a_{t-30}, ..., x_{t-1}, a_{t-1}, o_t)$ that takes as input the history of observation and outputs the action command (no explicit latent regression)? \n\n\nClaims/Statements that need to be revised and made clear:\n* The learned gait is a quasi-static gait and the rotation is only around $z$-axis. From the demo videos, it's safe to say that at any time, the object is in a relatively stable pose with multiple finger support. This is sufficient for rotating the objects around the $z$-axis, which is not a bad thing. I think this is also why the policy works for different objects. The fingers have good support on the object being rotated, the policy only needs to figure out when contact happens between the fingertips and object (which can be inferred from the history of observations) and then move the fingers one by one.  The point I wanna make is that this task is much easier than dynamic object reorientation in full $SO(3)$ space that many prior works explored. So I don't think it is appropriate to criticize prior works (on in-hand object manipulation) on the lack of generality or sim2real transfer without stating what this paper is actually doing. They are tackling in-hand manipulation tasks with different levels of difficulty.\n\n* The paper claims that the hand can rotate both convex and non-convex objects around $z$-axis. However, this is misleading. It is true that the objects used in the paper have different shapes. But what matters is the local geometry on which the contacts happen. For example, the paper uses bottle, paper towel, hourglass, sake cup, etc. At first glance, these objects look different. But because the paper only trains the hand to rotate objects around the vertical axis, the hand only needs to touch a very small convex (or almost convex) contour of the objects. And we can also see that on these objects (paper towel, hourglass, sake cup, shuttlecock, etc.), the rotation axis is carefully chosen such that other geometry of the objects does not affect the rotation. And the fingertips almost always work on the convex part of the objects. In this sense, these different objects used in the paper actually look quite similar. In addition, considering the fingertips of the Allegro hand are big, so the local contact geometry differences on the objects won't affect the motion as much. Therefore, I think it is misleading to claim that the policy can rotate both convex and non-convex objects. This should also be clearly discussed in the Limitation section.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I think the paper is working on an important problem. Dexterous manipulation is challenging. As a starting point, the paper addresses the problem of rotating objects around $z$-axis in a quasi-static way. The results are nice even though the method itself is completely from another published paper. However, I would require the authors to revise some misleading claims in the paper to make the writing more accurate. I am happy to raise my score if the writing is improved.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are not well addressed", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "issues": "Listed above.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)