DextrAH-G: Pixels-to-Action Dexterous Arm-Hand Grasping with Geometric Fabrics
Authors: Tyler Ga Wei Lum
Martin Matak
Viktor Makoviychuk
Ankur Handa
Arthur Allshire
Tucker Hermans
Nathan D. Ratliff
Karl Van Wyk
Venue: CoRL 2024 (2024)
Keywords: Dexterous Grasping
Geometric Fabrics
Reinforcement Learning
Teacher-Student Distillation
Sim-to-Real Transfer
TL;DR: DextrAH-G is a safe, continuously reacting pixels-to-action policy that achieves state-of-the-art dexterous grasping in the real world and was trained entirely in simulation using RL and teacher-student distillation with a geometric fabric controller

Abstract: A pivotal challenge in robotics is achieving fast, safe, and robust dexterous grasping across a diverse range of objects, an important goal within industrial applications. However, existing methods often have very limited speed, dexterity, and generality, along with limited or no hardware safety guarantees. In this work, we introduce DextrAH-G, a depth-based dexterous grasping policy trained entirely in simulation that combines reinforcement learning, geometric fabrics, and teacher-student distillation. We address key challenges in joint arm-hand policy learning, such as high-dimensional observation and action spaces, the sim2real gap, collision avoidance, and hardware constraints. DextrAH-G enables a 23 motor arm-hand robot to safely and continuously grasp and transport a large variety of objects at high speed using multi-modal inputs including depth images, allowing generalization across object geometry. Videos at https://sites.google.com/view/dextrah-g.

DextrAH-G: Pixels-to-Action Dexterous Arm-Hand Grasping with Geometric Fabries
Tyler Ga Wei Lum Stanford University
Martin Matak University of Utah
Viktor Makoviychuk NVIDIA
Ankur Handa NVIDIA
Arthur Allshire
University of California, Berkeley
Tucker Hermans
Nathan D. Ratliff NVIDIA
Karl Van Wyk"
NVIDIA and University of Utah
NVIDIA
(*),(*) indicates dual first and last author, respectively
Figure I: DextrAH-G (Dexterous Arm-Hand Grasping) continuously controls a dexterous robot to grasp and transport diverse range objects directly from streaming depth images.
Abstract: A pivotal challenge in robotics is achieving fast, safe, and robust dex- terous grasping across diverse range of objects, an important goal within in- dustrial applications. However, existing methods often have very limited speed, dexterity, and generality, along with limited or no hardware safety guarantees. this work, we introduce DextrAH-G, a depth-based dexterous grasping pol- icy trained entirely in simulation that combines reinforcement learning, geometric fabrics, and teacher-student distillation. We address key challenges in joint arm- hand policy learning, such as high-dimensional observation and action spaces, the sim2real gap, collision avoidance, and hardware constraints. DextrAH-G enables 23 motor arm-hand robot to safely and continuously grasp and trans- port a large variety of objects at high speed using multi-modal inputs includ- ing depth images, allowing generalization across object geometry. Videos bttps://sitess gwogle. wwwcccooiiiiiieeeeeeeeoo dextrah-g.
Keywords: Dexterous Grasping, Geometric Fabrics, Reinforcement Learning, Trachee-Sttdent Distillation, Sim-to-Real Transfer
1 Introduction
Grasping, fundamental skill coarsely mastered by two-year-olds [1], still poses challenge in robotics. Equipping machines with proficient grasping skill is paramount to automating aspects of logistics, manufacturing, space, and search-and-rescue among many other use cases. To date,
8th Conference Robot Learning (CoRL. 2024), Munich, Germany.

---

robot grasping has emerged as chief research problem with many advancements made. However, existing methods often have limited speed, dexterity, and reliability, especially with high-actuator- count embodiments. Moreover, partial observability and succinet coordination over a large number of motors in dexterous platforms pose additional challenges acquiring proficient grasping skill.
Existing approaches in dexterous robot grasping (see Section 2) mostly focus predicting grasp poses while relying on model-predictive control, path planning, or other motion generation tools to reach the grasp pose. While somewhat successful, this approach is not continuously reacting strategy, typically does not plan through all degrees-of-freedoon and ignores grasp modulation and coordinated finger-arm control post-grasp. Generating high-frequency control predictions from all available data sources (e.g. fusing all sensory traces including proprioception and vision) critical for reasoning through partially observable environments and improving skill performance.
Encouragingly, reinforcement learning in simulation at scale has enabled significant progress in both legged locomotion and in-hand manipulation even with high-dimensional observations like point clouds, high-ding high-dimensional action spaces, and gigh-freueeccy action rates. The ability to efficiently scale experience with continued policy optimization and domain randomization has singularly en- abled the creation of highly dynamic, complex control over dexterous robot platforms. A compara- tively smaller but growing body of work also applies RL in simulation scale advance the state of dexterous robot grasping in the real world. However, the exact methods to date typically lead to some combination of the following qualities: slow robot motion, inadequate or missing hardware safety guarantees, unnatural movements and postures, and limited generalization.
To address these challenges with real-world dexterous robot grasping, we propose DextrAH-G: a combined pixels-to-action fabric-guided policy (FGP) and geometrie fabric controller that achieves ofate-of-the-art grasping performance in the real world. The main contributions of this work include: 1) vectorized geometric fabric controller that creates an inductive bias for policy learning, avoids collision, upholds joint constraints, and shapes the behavior, 2) simulation-only RL training of privileged FGP over vectorized geometric fabrics that enables high-performance grasping of many different objects, 3) septh-based, multi-moda FGP distillation of the privileged FGP that replicates the original behavior and enables object position predictions and 4) zero-shot sim2real transfer of DextrAH-G with new ofate-of-the-art dexterous grasping performance on many diverse novel objects in the real world, significant progression towards grasp-anything skill in dexterous robotics.
2 Related Work
2.1 Dexterous Grasping
Dexterous grasping a research problem that has been extensively studied for decades. Traditional methods [2] use gradient-based or sampling-based optimizers maximize analytical grasp met- rics, such differentiable approximations of form closure [3], force closure [4], the min-weight metric [5], and the largest inscribed ball metric [6]. However, these works are typically limited to precision grasps using only the fingertips and require ground-truth object models.
In recent years, data-driven learning-based methods have emerged as a promising approach for dex- terous grasping of complex objects. Li et al. [7] and Wang et al. [3] create large-scale grasp datasets on diverse objects using approximate force closure optimization. Xu et al. [8] use these grasp datasets to learn grasp proposal generation network generate grasps conditioned on full full-point clouds and then train a goal-conditioned grasp RL policy to perform these grasps. Wan et al. [9] build on this method with a geometry-awarr curriculum and iterative seneralist-specialis learning. How- ever, their method requires full full-poiia -oiltpoint clouds of objects and has only been tested in simulation. Liu et al. [10] introduces novel geometric and spatial hand-object interaction representation to capture dynamic object shape features and the spatial relations between hands and objects during grasp- ing, and they use this representation to train a dexterous grasping policy using RL. However, when deployed in the real world, this representation requires ground-truth object models and requires reg- istering these models to the measured partial point cloud, limiting its generality. Agarwal et al. [11]

---

select gre-grasp position on a target object by matching the DINO-ViT features of a previously annotated object, then use a blind (uses only proprioception) grasping RI. policy with an eigengrasp action space that is trained entirely in simulation. Others trained point-clou-conditionee policy with RL in simulation using imagined hand point clouds as augmented inputs which was deployed in the real world on novel objects the same category Qin al. [12]. These two works are similar to ours except our policy uses depth images as input jointly select actions for both the arm and hand, allowing for more agile, coordinated, and generalized behavior across novel objects. See Appendix A for an extended discussion on related works RI. for robot control.
2.2 Policy Learning and Control
Learned policies for robots typically issue actions to simple controllers like joint-level proportional- derivative (PD) control or operational space control (OSC). Consequently, controller simplicity shifts the burden of discovering and imparting well-controledd behavior entirely to the learned pol- icy itself. Globally, well-controlled behavior has complex. multi-faceted priorities such as primary objective completion, various hardware constraints, collision avoidance, and other qualities like nat- ural movements. Discovering this rich behavioral conglomerate via learning is very challenging due to optimization locality, approximation and generalization errors inherent neural networks, behavior specification, exploration, and high-dimensional action spaces. This arduous path can be avoided by embedding many of these behaviors within more sophisticated control layers. Recently, fabric-guided policies (FGP) were trained over a geometric fabric via RL simulation to achieve new state-of-the-art in-hand cube reorientation performance the real world [13]. The geometric fabric itself handled joint hardware constraints, created an input action space for controlling the hand fingertips, and guided the fingertips towards making contact with the cube. Leveraging such rich controller promoted much simpler reward function predominantly geared towards primary task completion. Other works also mix sophisticated control and policy learning like Riemannian Mo- tion Policies in [14] and QP control in [15]. Since geometric fabrics have been shown to outperform RMPs [16, 171, DMPs [17]. Koopman Operator policies [18], and derive from strong theoretical analyses [19, 16, 20], we elect use geometric fabries within this work to construct A--trAH-G.
3 DextrAH-C. Dexterous Arm-Hand Grasping
We present DextrAH-G, a combined pixels-to-action FGP and geometric fabric controller that achieves dynamic and reactive dexterous grasping in the real world. The instantiated geometric fabric for DextrAH-G is the most feature-rich design to date that shapes posture and grasping be- havior, avoids environment and self-collision imposes joint constraints, and exposes efficient action spaces. Our controller facilitates sim2real by: 1) efficiently scaling for vectorized RL training while maintaining real-time loop rates for real-world deployment, and 2) enabling safe real-world deploy- ment even with delusional, hazardous policies. We now detail DextrAH-G by first describing the geometric fabric controller. Then, we discuss how we train our privileged teacher FGP using RL entirely in simulation. Finally, we describe how we distill this privileged FGP into depth FGP that we deploy zero-shot on hardware. Figure 2 shows our proposed framework.
3.1 Geometric Fabrics and Fabric-Guided Policies (FGPs)
Geometric fabrics generalize the behavior of classical mechanical systems and, thereby, can be used to model controllers with design flexibility, composability, and stability without the loss of modeling fidelity. Behavior expressed by a geometric fabric follows the form
My 4 ) + fy (r) + fx( 0
where M, E Raxn is the dositive-definite system metric (mass), which captures system prioritiza- tion, fy E R is nominal path generating geometric force, and (a) E I is an additional driving force of some action E T q,: qf. q R²" are the position, velocity, and acceleration of the fabric. This fundamental equation produces acceleration q,, which evolves the fabric state q
3

---

Stage 1: Teacher Privileged FGP Training (Reinforcement Learning)
Asymmetric
Full State
Forces
Nolsy Object Pose
Object ID
CPOO
Proprio
Position
Simulation Hz)
RNW
My (fy [(a)
Targets
4 4y stop Gr)
Action
Geometric Fabric (60 Hz)
Privileged FGP (15 Hz))
Stage 2: Student Depth FGP Training (Policy Distillation)
Online DAgger
Object
Los
Position
CNN
MLP
Depth
Image
RNN
-Kayi
Predicted
Simulation (60 Hz)
Proprio
Goal
Depth FOP (15 Hz)
Pasition Swd
Joint PD Targets
4
Naisy Object Pose
Legend
Object ID
Crilic Inputs
Privileged FGP Inputs
Proprio
Position
MLP
Lipetion
Action
RNN
Object Position
-M (f, + fa(a))
Depth Imputs
dy stop'ty 4 (r))
Action
Privieged FOP (15 HZ)
Frozen Model Weights
Geometric Fabric (60 Hz)
Stage 3: Real World Deployment
Opsket Ngasz
Depth Image
HNN
Proprio
Goal Position
Action
Depth FGP (15 Hz]
Predicted
Pasilion Xaño
(1000 Hz)
4 My¹ (fy + (m))
Joint PD Targets 9
step(sp 4,.
Finel
Action
Geometric Fabric (60 HZ)
a yina!
State Machine
Figure 2: We train a privileged fabries -ubries-guided policy (FGP) using RL (top), distill the privileged FGP into depth FGP to predict the teacher's actions and object position (middle), and deploy DextrAH-G with state machine in the real world for bin packing (bottom). See Table 3 for details.
and qr over time through numerical integration. One can see that f, influences d and thereby, the fabric state. For in-depth discussion of geometric fabrics, we refer the reader to prior works [19, 16, 20, 13]. See Appendix B.1 for details how geometric fabric connects with robot.
We use geometric fabric controller for four primary reasons: 1) avoiding undesirable collisions, 2) creating an inductive bias via the exposed action space that simultaneously guides policy exploration and favorably shapes the overall robot motion, 3) respecting joint constraints (see Appendix B.3), and 4) maintaining the robot posture promote kinematic manipulability (see Appendix B.4).
Environmental and self-collision avoidance are handled through geomet- ric fabric term and forcing fabric term. The geometric term is tuned to dominate the collision avoidance behavior with speed-invariant paths, while the forcing term prevents penetration near the collision boundary. First, we model the geometry of the robot as a collection of spheres and use the forward kinematics the arm to map the robot configuration to the origin of every sphere attached the robot body, x = hy (q) R We define ñ; r--X R as the direction from the sphere |||:-]]
Collision Avoidance:
point to the closest point collision body E g (collision objects other body spheres) and

---

max(dmin dmin, d.) E R+ as a sower-bounded distance, where dmin E R and d; E IR is the signed distance between the body sphere and collision body i (Figure 5 for visualization).
d, = -max
The geometric acceleration is x kg | and the forcing acceleration is x k bic, where kgg ky E R+ are gains and b E I + is a damping scalar. 8, Σ. 1 celeration response per sphere away from collision, and &, ||all|
Σ. is base Se the normalized base accel-
n;
eration. The geometric and forcing terms both have metric design M
Ma. where 3 R+ asto
is gain, and d min,(d.) Mr Σ a. 2 i, is a base metric response per sphere, and M. M, = [M.Il is the normalized metric, which maintains its Eigenspectrum (directions of impor- tance). 8, = lanh(-an - (v2) + 1) a smooth velocity gate that goes high when this sphere moving towards collision body i, where an 02 R+ are gains and v; -x f is the signed impact speed that is negative when moving towards collision body (see Appendix B.2 for details).
During grasping, human finger motions exhibit highly correlated movements that do not fully exercise all the degrees of actuation afforded by our hands. These grasps can most broadly be categorized into power and precision grasps [21]. Absent intricate in-hand manipulation, grasp- ing motion can be concisely captured by movement on a lower dimensional manifold. Constraining policy's action space an eigengrasp manifold has been shown to improve grasp policy learn- ing [11]. We create such manifold for the Allegro hand by retargeting human grasping motion data to the Allegro (see Appendix D for retargeting details) and apply principal component analysis to the motion dataset. Let A E R5x16 be the first five principal components from PCA and define A = = A] E 1R5223 then the taskmap from the full robot configuration space to PCA space is Aq E R$. Within this taskmap, we define an attraction fabric term. The metric M(x) = ml is
Action Space:
a constant isotropic mass, where m E IRt x = - Ka tanh(aa (0-eaa/aX/XXxeettrrrriee Xarretrrget x Xyoa durget bx, I)) || Xxce turgetll
where ka, aa E Rt are gains and tara,targes tarett as dimensional action space for the hand (Figure 7 for example motions).
E R is a target position in this space. We use
To coordinate finger control with arm control, create another action space that controls the pose of the palm. We accomplish this by creating new taskmap that uses forward kinematics to map to 7 threc-dimensional points attached to the palm, stacked into 21 dimensional space. The attraction fabric in this space is the same as designed for the hand, but scales to the full 21 dimensions with the goal action position Xg E 2R We create a dimensional action space for the arm that consists of target palm position Xf.target E R33 and target palm orientation in Euler angles ff,target TR3 These quantities are transformed to 3D point targets for all 7 palm-fixed points, i.e., Xg. and issued to the fabric. Across the full robot, the action space is 11 dimensional. This action space resulted in faster training and more natural grasping behavior (see Appendix C).
3.2 Teacher Privileged FGP Training (Reinforcement Learning)
Next, cast dexterous grasping reinforcement learning problem and train a privileged-state teacher policy in simulation to adeptly grasp 140 different objects (additional details in Appendix E). Since the geometric fabric action space ensures that the robot will execute safe and natural behaviors, our reward design centers entirely around fingertip-object contact and lifting the object to a goal. This simplifies reward engineering and reduces reward hacking behavior from the policy.
Asymmetric Actor Critic: Although our control policies will not have access privileged sim- ulation state information when deployed in the real world, we can still use privileged information to accelcrate training in simulation. We use Asymmetric Actor Critic training [22], in which our critic Vss) is given all privileged state information s and our teacher policy Tpriesteged (ppriittgaa)) is provided an observation Oprivsleged, which is a limited subset of this privileged state information. We believe this prevents the teacher policy from learning behavior that is heavily reliant on accurate privileged state information, which would not be learnable by the student policy with real-world perception inputs. However, the critic can still leverage this information to provide more accurate value estimates, improving the speed and quality of teacher policy training.
5

---

We define the teacher policy's 's observation as Oprinileged [Orobot, xgoal: O Orooot includes the espace position q E R espace velocity á E RR (N₂ = 23), positions of three points on the palm [xpalm: Xpalm xxpalm=:xpalmm> E R333 positions of the Ningers fingertips Xinnerttiss E IR-Ngungera X3, and the fabric state [qr- Qr qy] E R Xgoal E 12 is the goal object position. Oobj includes the noisy object position Xobj E R and quaternion Jobj E IR (more details below), and the object one-hot embedding E {0, 1] ]/Mokeests, where Nobjects 140 is the number of objects in the training dataset (see Appendix E.7 and E.8).
Observation, State, and Action Space:
We define the critic's input state as s = priestteged Spriniteged)- Spriesteged contains privileged state information including robot joint forces fdos E IR,, fingertip contact forces fingers E R-Nringer x3, true object position Xobj E IR3, true object quaternion qobj E IR+, true object ve- locity Vouy E R and true angular velocity Wobs R This additional information allows the critic to make more accurate value predictions, which helps the policy learn more quickly.
We define the teacher policy's action a as inputs to the underlying geometric fabric, where a [x (orgett /taaggt:: Xarget E IR' where Xtrrgttt E R is the target palm position, ftarget E R& is the target palm orientation in Euler angles, and Xpca, E R is the target PCA position for the fingers. The fabric is integrated at 60 Hz and the simulation steps at 60 Hz. The teacher policy runs at 15 Hz, so actions are repeated for intermediate timesteps.
Environment Modifications for Robust Grasping: RL policies often converge to unnatural so- lutions that work well in simulation, but fail to transfer to the real world (see Appendix E. for frail grasping behavior.) We learn robust grasping behavior by introducing the following environ- ment modifications. Random Wrench Perturbations: We apply random wrenches that move and rotate the object in unpredictable ways (see Appendix E.3). This forces the policy learn grasps that are robust to exogenous perturbations. Pose Noise: We add uncorrelated and correlated noise to the object pose observation (see Appendix E.4). This gives incentive to learn grasping behavior that opens the hand wider than typically needed when approaching the object to reduce unexpected contact and account for uncertainty in position and geometry. Friction Reduction: We reduce the default coefficient of friction of the object to j = 0.7, mitigating grasping behavior that is overly reliant on friction. Domain Randomization: We employ domain randomization across simulation parameters learn policies that are robust across spectrum of dynamics (see Appendix E.5).
3.3 Student Depth FGP Training (Policy Distillation)
We use the teacher-student framework and distill our expert to student policy that can be deployed in the real world using online version of Dagger [23]. This distillation results in pixels-to- action policy that uses continuous image input at 15 Hz to perform reactive dynamic grasping in the real world. During distillation, the student deepth (Odepth) + (å, Robj) receives an observation deepth = [rohht Xgoat: I]. where I E [0.5, 1.5) 160x 120 m is a raw depth image. It produces actions E I and object position predictions Rayj E IR3 (Xoty is used by the state machine during real-world deployment). The student is trained with supervision loss L = Laction + 3Bppss where Laction = l| a||2 and Lyms ||antj Xabi | where are actions predicted by the teacher Tprieileged and Xabj are ground-truth object positions from the simulator, and 8 0.1 (see Appendix F). Since the policy fuses depth and proprioceptive signals together, Xobj is more accurate through occlusions and facilitates its usage in state machine. We augment the simulated depth readings with noise (see Appendix F.1) to better match the distractors present in the real world.
4 Experiments
4.1 Simulation
We evaluate wdepth on the 140 training objects and report 99% success rate on average per batch of finished environments, matching Tprisileged performance. Next, we evaluate deepth per object and report 80% success rate (versus 85% for Aprivileged) on average per object with an average successful episode lasting 4 seconds (see Appendix F.4). The 19% gap stems from environments

---

Kuka liwa
Alleero Hand
Realsense
Figure 3: The robot platform consists of an Allegro hand mounted to Kuka LBR iiwa arm, one Intel Realsense D415 camera, a work table, and a bin to drop grasped objects.
Object
Pitcher
Pringles
Coffee
Container
Cup
Cheezil
Cleaner
Brick
Spam
Airplane
DextrAH-G (Ours)
80%
100%
100%
100%
80%
100%
100%
100%
100%
100%
60%
DexDiffuser [25]
60%
60%
80%
100%
20%
ISAGrasp [26]
60%
40%
80%
80%
Matak [27]
67%
100%
67%
0%
0%
100%
100%
0%
Table 1: Single object grasp success rates for standard test objects out of 5 trials per object. Success
rates for baselines are as reported in literature.
being reset after successful grasp in the first experiment. This leads easier objects being grasped more often, resulting in increased success rate. Overall, Tdepth nearly matches the performance of π priiiiteedd enabling sim2real transfer as discussed in the subsequent section.
4.2 Real- World
Hardware Setup: The physical setup consists of an Allegro Hand mounted to a Kuka LBR iiwa arm and one Intel Realsense D415 camera rigidly mounted to the table (see Fig. 3). This robot has 23 independent motors and a single camera stream for control policies. Both the arm and the hand have an underlying joint PD controller which operates kHz for the arm and 333 Hz for the hand. These operate in two different ROS 2 nodes and listen for joint commands. The geometric fabric runs at 60 Hz in another ROS 2 node, which receives actions and outputs joint commands at 60 Hz. Finally, deepth runs in yet another separate node which receives all the necessary inputs and outputs actions at 15 Hz. The separation in FGP and fabric nodes enables persistent controller command over the robot behavior regardless of the state of the FGP node or FGP model.
Single Object Grasping Assessment: popular assessment protocol for dexterous grasping that of quantifying single-object success rates. We conduct this procedure across 11 objects that exist in standardized object sets in [24] and adopted by others in grasping research. The pro- cedure consists of placing object in five different poses the table and deploying the robot grasping behavior. The grasp success rate is calculated across five trials for each object. We run DextrAH-G continuously until the grasp succeeds or we experience an irrecoverable failure. This allows DextrAH-G to aggregate interaction data with its recurrent structure to improve and adapt its actions over time, resulting in new state-of-the- grasping success rates as reported in Table This grasp assessment protocol is not a very complete assessment of grasping performance as does quantify grasping speed and does consider grasping behavior within the context of full tranassotr application. As such, we propose a bin packing protocol.
The bin packing testing protocol quantifies the performance of contin- uously grasping and transporting a larger variety of objects. Unlike the single-object assessment, this test captures performance within full application context. Specifically, the robot is charged with continuously grasping over 30 different objects one-at-a-time and transporting them bin placed to the side of the robot (see Figures 3, 4). simple state machine governs the whole pro- cess and uses Xot to transition from grasping to transportation (see Appendix G). We propose three
Bin Packing Assessment:
7

---

Figure 4: DextrAH-G robustly grasps and transports diverse and novel objects in the real world.
primary metrics to quantify grasping performance: 1) consecutive success (CS) the number of consecutively successful object transports, 2) cycle time the time required for the robot to grasp an object, transport it, and return ready position, and 3) success rate. Across eight different runs, DextrAH-G achieved CS mean and 95% confidence interval of 6.56 + 2.41 objects. This shows that DextrAH-G is capable of transporting several objects in row before failing (e.g.. object falls off the table). Moreover, DextrAH-G also managed cycle time mean and 95% confidence interval of 10.66 + 0.84 seconds, or 5.63 picks-per-minute (PPM) on average. Finally, DextrAH-G successfully grasped and transported all objects with 87% success across 256 attempts. DextrAH- G's combined celerity and reliability significantly advances the statc-of-the art in dexterous robot grasping, bringing real-world utility nearer. For reference, we estimate human solve rate of 16.53 PPMs for this task based on the Boothroyd-Dewhur ing time, and return time [28]. Under continuous operation, DextrAH-G's cycle time is already compelling performance point for practical usage and expect to improve it in the near future. For additional performance analysis, see Appendix I.
tables with 1.13 grasp time, 1.5 s reposition-
5 Limitations
Limitations of DextrAH-G are as follows. First, the overlying FGPs issue goals in the PCA taskmap for controlling the fingers. While this was intentionally chosen to focus grasping behavior, does limit the kinematic dexterity of the robot. Second, some amount of obstacle avoidance behavior should ideally be learned based on sensory inputs to reduce dependency on model-based behaviors. Additionally, RL still struggles to explore near high-cost regions. For example, the fabric's obstacle avoidance keeps the robot from significant collision with the table, but this behavior also makes effective exploration in these regions more difficult resulting in reduced performance for low-profile objects. Improving exploration strategies, the RL algorithm itself, learning collision avoidance via a curriculum are possible avenues forward. Finally, DextrAH-G can only handle one object in the scene at a time and likely needs changes (e.g. segmentation) to work effectively with clutter.
Conclusion
DextrAH-G is a gegh-performacc dexterous grasping policy with depth inputs trained entirely in simulation and deployed in the real world. To achieve this, we combined RI., online distillation, and a vectorized, feature-rich geometric fabric controller. The fabric itself ensures hardware safety and exposes an action space with a strong inductive bias. RL leverages the geometric fabric learn privileged FGP for grasping many different objects which is then used to directly train a depth FGP via online distillation. DextrAH-G successfully grasps and transports a large variety of novel objects in the real world, bringing real-world application closer to reality. Critically, over the many hours of testing AH-G (and a variety of ill-behaved FGPs), no hardware was damaged.

---

Acknowledgments
This work is supported by NVIDIA, DARPA under grant N66001-19-2-4035, NSF Award #1846341, and NSERC Award #526541680.
References
[1] M. Paré and C. Dugas. Developmental changes in prehension during childhood. Experimental brain research, 125:239-243 247, 1999.
[2] A. Miller and P. Allen. Graspit! versatile simulator for robotic grasping. IEEE Robotics & Automation Magazine, 11(4):1 2004. 100.1000.....0..11.......
[3] R. Wang, J. Zhang, J. Chen, Y. Xu, P. Li, T. Liu, and H. Wang. Dexgraspnet: large- scale robotic dexterous grasp dataset for general objects based on simulation. arXiv preprint arXiv:2210002699 2022.
[4] A. Wu, M. Guo, and K. Liu. Learning diverse and physically feasible dexterous grasps with generative model and bilevel optimization. In K. Liu, D. Kulic, and J. Ichnowski, editors, Conference on Robot Learning, CoRL 2022, 14-18 December 2022, Auckland, New Zealand, volume 205 of Proceedings of Machine Learning Research, pages 1938-1948. PMLR, 2022. URL www.thttpp/.oo http///poceeenng...ee/e//
mlr /
wwd3b html.
[5] A. Li, P. Culbertson, J. W. Burdick, and A. Ames, Frogger: Fast robust grasp gener- ation via the min-weight metric. ArXiv, abs/2302.13687, 2023. URL https://agg semanti centiicccllar
[6] C. Ferrari and J. Canny. Planning optimal grasps. Proceedings 1992 IEEE International Conference on Robotics and Automation, pages 2290-2295 vol.3, 1992. doi:10.1 109/ROBOT. 1992.219918.
[7] Li, T. Liu, Y. Li, Y. Zhu, Y. Yang, and S. Huang. Gendexgrasp: Generalizable dexterous grasping. arXiv preprint 20Xiv:2210.0000 2022.
[8] Y. Xu, W. Wan, J. Zhang, H. Liu, Z. Shan, H. Shen, R. Wang. H. Geng, Y. Weng, J. Chen, et al. Unidexgrasp: Universal robotic dexterous grasping via learning diverse proposal generation and goal-conditioned policy. arXiv preprint arXiv:23033. 00938, 2023.
[9] W. Wan, H. Geng, Liu, Z. Shan, Y. Yang, L. Yi, and H. Wang. Unidexgrasp++: Improving dexterous grasping policy learning via geometry-awan curriculum and iterative generalist- specialist learning. arXiv preprint xxXiv:2304.00464 2023.
[10] Q. Liu, Y. Cui, Z. Sun, H. Li, Li, L. Shao, Chen, and Q Ye. Dexrepnet: Learning dex- terous robotic grasping network with geometric and spatial hand-object representations. 2023 IEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pages 3153- 3160, 2023. URL https:://// wwtp.sssmatiischllaa...o.. wowpp/uuurro//oouuuaoaaaaaaaaala ID :27 622709.
[11] A. Agarwal, S. Uppal, Shaw, and D. Pathak. Dexterous functional grasping. In 7th Annual Conference Robot Learning, 2023. URL https: /ttp::://////eeeeeeeee net. formmmid- 93qz1k6_6h.
[12] Y. Qin, B. Huang, Z.-H. Yin, H. Su, and X. Wang. Dexpoint: Generalizable point cloud reinforcement learning for sim-to-real dexterous manipulation. Conference Robot Learning (CoRI.), 2022.
[13] K. Van Wyk. A. Handa, V. Makoviychuk, Y. Guo, A. Allshire, and N. D. Ratliff. Geometric fabrics: safe guiding medium for policy learning. arXiv preprint urXiv:2405.022550 2024.
[14] A. Li, C.-A. Cheng, M. A. Rana, M. Xie, K. Van Wyk, N. Ratliff, and B. Boots. Rmp2: A structured composable policy class for robot learning, arXiv preprint 2rX.v:2103.05922. 2021.

---

[15] F. Wirnshofer, P. S. Schmitt, G. von Wichert, and W. Burgard. Controlling contact-rich manip- ulation under partial observability. In Robotics: Science and Systems, 2020.
[16] K. Van Wyk, M. Xie, A. Li, M. A. Rana, B. Babich, B. Peele, Q. Wan, I. Akinola, B. Sun- daralingam, D. Fox, al. Geometrie fabrics: Generalizing classical mechanics to capture the physics of behavior. IEEE Robotics and Automation Letters, 7(2)3322--3209, 2022.
[17] M. Xie, A. Handa, S. Tyree, D. Fox, H. Ravichandar, N. D. Ratliff, and K. Van Wyk. Neu- ral geometric fabrics: Efficiently learning high- figh-dimensional policies from demonstration. In Conference on Robot Learning, pages 1355-1367. PMLR, 2023.
[18] Y. Han, M. Xie, Y. Zhao, and H. Ravichandar. On the utility of koopman operator theory in learning dexterous manipulation skills. In Conference on Robot Learning, pages 106-126. PMLR, 2023.
[19] N. D. Ratliff, K. Van Wyk, M. Xie. A. Li, and M. A. Rana. Generalized nonlinear and finsler geometry for robotics. In 202/ IEEE International Conference on Robotics and Automation (ICRA), pages 10206-10212. IEEE, 2021.
[20] N. Ratliff and K. Van Wyk. Fabrics: A foundationally stable medium for encoding prior experience. arXiv preprint 23Xiv2309.07366. 07368, 2023.
[21] M. R. Cutkosky et al. On grasp choice, grasp models. and the design of hands for manufactur- ing tasks. IEEE Transactions on robotics and automation, 5(3):269-21) 1989.
[22] L. Pinto, M. Andrychowicz, Welinder, W. Zaremba, and Abbeel. Asymmetric actor critic for image-based robot learning. In H. Kress-Gazit, S. S. Srinivasa, T. Howard, and N. Atanasov, editors, Robotics: Science and Systems XIV, Carnegie Mellon University, Pits- burgh, Pennsylvania, USA, June 26-30, 2018, 2018. ptt.t/......ll......
XIV.008 URL
http: / www wtw.:roooooccppccceiinggoo..oeee http://cwweinnn..oog/rrrseeeeeee www.:rrgggrrsooeee p/0
[23] S. Ross, G. Gordon, and D. Bagnell. A reductioo of imitation learning and structured prediction to no-regret online learning. In G. Gordon, D. Dunson, and M. Dudík, editors, Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, volume 15 of Proceedings of Machine Learning Research, pages 627 -655, Fort Lauderdale, FL, USA, 11-13 html.
Apr 2011 PMLR. URL https: /ttctps://ppoceeediiiie http///ecccee....eeee/e/ mlr htteess//ll//oo/s//
[24] B. Calli, A. Walsman, A. Singh, S. Srinivasa, P. Abbeel, and A. M. Dollar. Benchmarking in manipulation research: The ych object and model set and benchmarking protocols. arXiv preprint arXiv: 1502.03143 2015.
[25] Z. Weng, H. Lu, D. Kragic, and J. Lundell. Dexdiffuser: Generating dexterous grasps with diffusion models. arXiv preprint arXiv:2402.02900 2024.
[26] Z. Q. Chen, K. Van Wyk, Y.-.. Chao, W. Yang, A. Mousavian, A. Gupta, and D. Fox. Learning robust real-world dexterous grasping policies via implicit shape augmentation. arXiv preprint arXiv:2210. 136388 2022.
[27] M. Matak and T. Hermans. Planning visual-tactile precision grasps via complementary use of vision and touch. IEEE Robotics and Automation Letters, 8(2):768-777 775, 2022.
[28] G. Boothroyd, P. Dewhurst, and W. A. Knight. Product design for manufacture and assembly. CRC press, 2010.
[29] Wu, A. Escontrela, D. Hafner, Abbeel, and K. Goldberg. Daydreamer: World models for physical robot learning. In K. Liu, D. Kulic, and J. Ichnowski, editors, Conference on Robot Learning, volume 205 of Proceedings of Machine Learning Research, pages 2226-222 22442, 14
18 Dec 2023.
10

---

[30] R. Jitosho, T. Lum, A. Okamura, and K. Liu. Reinforcement learning enables real-time plan- ning and control of agile maneuvers for soft robot arms. 2023.
[31] J. Tan, T. Zhang, E. Coumans, A. Iscen, Y. Bai, D. Hafner, S. Bohez, and V. Vanhoucke. Sim- to-real: Learning agile locomotion for quadruped robots. In Proceedings of Robotics: Science and Systems, 2018.
[32] A. Boeing and T. Bräunl. Leveraging multiple simulators for crossing the reality gap. In International Conference on Control Automation Robotics Vision, pages 1113-1119, 2012.
[33] S. Koos, J.-B. Mouret, and S. Doncieux. Crossing the reality gap in evolutionary robotics by promoting transferable controllers. In Conference on Genetic and Evolutionary Com- putation, GECCO '10, page 119-126. Association for Computing Machinery, 2010. ISBN 9781450300728.
[34] X. B. Peng, M. Andrychowicz, W. Zaremba, and Abbeel. Sim-to-real transfer of robotic control with dynamics randomization. In IEEE International Conference on Robotics and Automation, pages 3803-3810. IEEE, 2018.
[35] J. Dao, H. Duan, K. R. Green, J. W. Hurst, and A. Fern. Learning to walk without dynamics randomization. In 2nd Workshop on Closing the Reality Gap in Sim2Real Transfer for Robotics (Robotics: Science and Systems), 2020.
[36] A. Kumar, Z. Fu, D. Pathak, and J. Malik. RMA: Rapid Motor Adaptation for Legged Robots. In Proceedings of Robotics: Science and Systems, 2021.
[37] Miki, J. Lee, Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter. Learning robust per- ceptive locomotion for quadrupedal robots in the wild. Science Robotics, 7(62):eab 2022.
[38] N. Rudin, D. Hoeller, P. Reist, and M. Hutter. Learning to walk in minutes using massively parallel deep reinforcement learning. In Conference on Robot Learning, volume 164 of Pro- ceedings of Machine Learning Research, pages 91-100, 2022.
[39] J. Siekmann, K. Green, J. Warila, A. Fern, and J. Hurst. Blind bipedal stair traversal via sim- to-real reinforcement learning. In Proceedings of Robotics: Science and Systems, 2021.
[40] O, M. Andrychowicz, B. Baker, M. Chociej. R. Józefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, J. Schneider. S. Sidor, J. Tobin, P. Welinder, L. Weng, and W. Zaremba. Learning dexterous in-hand manipulation. The International Jour- nal of Robotics Research, 39(1):3 30, 2020.
[41] OpenAl, I. Akkaya, M. Andrychowicz, M. Chociej. M. Litwin, B. McGrew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribas, J. Schneider, N. Tezak, J. Tworek, P. Welinder, L. Weng, Q. Yuan, W. Zaremba, and L. Zhang. Solving Rubik's cube with a robot hand. arXiv preprint 191001991.001333 2019.
[42] A. Handa, A. Allshire, V. Makoviychuk, A. Petrenko, R. Singh, J. Liu, D. Makoviichuk, K. Van Wyk, A. Zhurkevich, B. Sundaralingam, Y. Narang, J.-F. Lafleche, D. Fox, and G. State. Dextreme: Transfer of agile in in-hand manipulation from simulation to reality, arXiv preprint arXiv:22 13702, 2022.
[43] K. Rao, C. Harris, A. Irpan, S. Levine, J. Ibarz, and M. Khansari. Rl-cyclegan: Reinforcement learning aware simulation-to-real. In /EEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
[44] E. Tzeng, C. Devin, J. Hoffman, C. Finn, P. Abbeel, S. Levine, K. Saenko, and T. Darrell. Adapting Deep Visuomotor Representations with Weak Pairwise Constraints, pages 688-703. Springer International Publishing, 2020. ISBN 978-3-030-4308944
11

---

[45] Т. Chen, M. Tippur. S. Wu, V. Kumar, E. Adelson, and P. Agrawal. Visual dexterity: In- hand reorientation of novel and complex object shapes. Science Robotics, 8(84))aade9244, 2023. doi: 10.1 http://ccooootic.cac222444 URL https: http://wwwccccc.rrrgeeeeeee//
10. s/c 4
[46] A. Agarwal, A. Kumar, J. Malik, and D. Pathak. Legged locomotion in challenging terrains using egocentric vision. In Conference on robol learning, pages 403-415. PMLR, 2023.
[47] Y.W. Chao, W. Yang, Y. Xiang, P. Molchanov, A. Handa, J. Tremblay, Y. S. Narang. K. Van Wyk, U. Iqbal, S. Birchfield, al. Dexycb: A benchmark for capturing hand grasp- ing of objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9044-9053, 2021.
[48] A. Handa, K. Van Wyk, W. Yang, Liang, Y.-W. Chao, Q. Wan, S. Birchfield, N. Ratiff, and D. Fox. Dexpilot: Vision-based teleoperation of dexterous robotic hand-arm system. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 9164-9170. IEEE, 2020.
[49] A. Petrenko, Allshire, G. State, A. Handa, and V. Makoviychuk. Dexpbt: Scaling up dexterous manipulation for hand-arm systems with population based training. CoRR, 12x/2305.12127, 2023. URL https: /ooi org/10 48550/arXia 2305. 12127.
[50] Haykin. Neural networks: comprehensive foundation Prentice Hall PTR, 1994.
[51] S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Comput., 9(8): 1735-1780, nov 1997. ISSN 0899-76677. doi: 10. 10..1.1///ncco.... 19t5//9....999.9.... 1997.9.8..733... URL https:
//doi. org. 10. 11 62 neco. 1997. 9. 8.8.8.1. 8735..
[52] A. Handa, A. Allshire, V. Makoviychuk, A. Petrenko, R. Singh, J. Liu, D. Makoviichuk, K. Van Wyk, A. Zhurkevich, B. Sundaralingam, et al. Dextreme: Transfer of agile hn-hand manipulation from simulation to reality. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 5977-5984. IEEE, 2023.
[53] Dawson-Haggerry et al. trimesh. URL httttttss//ttimmhhh...
[54] V. Makoviychuk, Wawrzyniak, Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, and G. State. Isaac gym: High performance gpu-based physics simula- tion for robot learning. arXiv preprint arXiv:2 10470, 2021.
[55] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint 1 2017.
[56] D. Makoviichuk and V. Makoviychuk. rl-games: A high-performance framework for rein- forcement learning,
com www//omm//nnce 88 Seenys88/rl_gameee rll_amms,, 2022.
[57] A. Handa, T. Whelan, McDonald, and A. Davison. benchmark for rgb-d visual odom- etry, 3d reconstruction and slam. In 2014 IEEE international conference Robotics and automation (ICRA), pages 1524-1531. IEEE, 2014.
[58] J. Werbos. Generalization of backpropagation with application a recurrent gas mar- ket model. Neural Networks, 1(4):339-366, 1988. ISSN 0893-6080. doisttps://doiorr//10. 0016/0893-6080088990000 URL http:///wwwwceeeeeeeeeeeeeece/e/ b
12

---

Appendix
A Additional Related Works on RL for Controlling Physical Robots
Reinforcement learning is data-driven approach to policy learning that excels in developing adap- tive behaviors for complex, contact-rich environments, which are traditionally challenging model with high accuracy. A prevalent method for using reinforcement learning on robots is to train policy entirely in simulated environment and then deploying these policies zero-shot in the real- world Wu et al. [29]. However, naively deploying these policies in the real world often results in poor performance. A common reason for this discrepancy can be from differences in sensing and actuation dynamics between the simulation and actual environments Jitosho al. [30], Tan al. [31], Boeing and Bräunl [32], Koos et al. [33]. To mitigate this issue, domain randomiza- tion be employed during training. enhancing the robustness of the policies against inaccuracies in simulation [34]. This technique has proven effective in various robotic applications, including legged locomotion [35, 36, 37, 38, 391, in-hand manipulation [40, 41, 42], and robotic grasping tasks [34, 43, 44J. Another challenge is that policies trained in simulation may require privileged information that is not accessible the real world, such as object pose and velocity, which does not allow them to be deployed in the real world. Therefore, use the real- observations from scratch or use a teacher with privileged information and then distill it into student policy that uses real-world observations. Qin et al. [12] train a point-cloud- conditioned policy with RL in simulation, and they find that they need to use imagined hand point clouds as augmented inputs to overcome challenges with object-hand occlusion in the real world point cloud. Chen et al. [45] train a teacher RL policy for in-hand object reorientation with priv- ileged information in simulation, use DAgger to distill this into a point-cloud conditioned student policy, and then deploy the system in the real world. Teacher-student distillation also prevalent in legged locomotion, where privileged experts are distilled into policies that operate on corrupted elevation scans [37, 46]. In our work, we utilize domain randomization improve robustness to spectrum of dynamics, train privileged teacher policy on compressed state information, and perform online distillation to train a student policy that operates on noisy depth images.
common to either train policies to
B Geometric Fabrics for Robot Control
B.1 Robot Control via Geometric Fabrics
Geometric fabrics is an artificial dynamical system that can be connected with real robots through a torque control law. Industrial, collaborative, and hobbyist robots commonly adopt torque laws that expose joint position and velocity action space, e.g., joint-level PD control. These controllers track the joint position and velocity targets over time, resulting in closed-loop tracking control. The fabric state above can be used directly as these joint position and velocity targets. Thus, the two dynamical systems become coupled and positively correlated as discussed in [13]. In effect, the real robot's dynamics are shifted to follow the artificial dynamics closely. Note, in our implementation, the fabric forward integrated with an approximate second-order Runge- Kutta scheme in [13] at 60 Hz and we always set the velocity targets O.
B.2 Collision Avoidance Details
Next, provide more detailed explanation about the formulation of the collision avoidance base metric response per sphere M, Σ, ž. , @ ú; The outer product n; @ ñ; creates matrix with an eigenvector along ñ. In essence, this matrix will give high priority to actions along ñi since that is the direction of importance. Building the complete base metric as a weighted sum over outer products of all ñ means that the resulting matrix will have eigenvectors along all ñi, capturing all directions of importance (directions that point prospective collision with collision body i).
13

---

B.3
Imposing Joint Constraints via Geometric Fabrics
First, since a fabric is a second order controller, joint acceleration and jerk limits can be handled in closed-form. We adopt the same technique as covered in [13] by solving the quadratic program:
= (ā. - q) " a a 7
(2)
2
where R Since â, (M, + ++ll) -fy, we can see that as co, | | + 0. Thus, single a can be found that drives every joint acceleration under its limit, i.e., q V where q; the 5thh joint acceleration limit. New joint acceleration limits, q. can be calculated that satisfies both the original acceleration limits and jerk limits, q, as
Arq 2Q
q min q
where At is the integration timestep. Thus, at every evaluation of d single a can be calculated that simultaneously satisfies all acceleration and jerk limits of the robot.
We also impose the robot's 's joint positional limits via the fabric by adopting the same joint repulsion term in [13]. Briefly, we construct an upper joint limit task space, x
q. and lower joint limit task space, x = q - q. where q and are the upper and lower joint positional limits, respectively. The fabric term in these spaces consists of the metric, M(x) diag (max(-sgn(xx). 0) fi where Ko R is constant gain, and acceleration x g bx. where g R (n is the joint space dimensionalityy. Effectively, the system accelerates away from a joint limit with increasing priority as distance to the limit decreases.
x
B.4 Posture Control
Since the fabric's exposed action space has fewer dimensions than controlled joints of the robot, must resolve redundancy issues. This is easily accomplished by following the configuration space geometric attractor as covered in [13]. Specifically, we create a geometric attraction term in the full joint space of the arm with metric, M(() = mI is a constant isotropic mass, where m E R+ = =- |22 tanh[(xa/|x - x, II) x-x,, x where hi E R+ is constant attraction gain, da is constant sharpness parameter, and X., is target state in this space. This is almost exactly the same design as the other two attraction terms, except the acceleration is homogeneous of degree two in velocity (HD2). This enables the fabric to guide the full robot movement towards Xg in configuration space, but not prevent the convergence Xg the PCA and pose taskmaps. The introduction of this term makes the fabric full rank, and furthermore, configuration. This specific target promotes kinematic manipulability by allowing the palm to move close to the table and robot. Without it, the overlying FGPs struggle in grasping objects closer to the robot base.
set X, to elbow-out, fingers-curled
C Cspace Fabric
Our initial design for DextrAH's geometric fabric (called espace fabric) allowed for issuing joint position targets as the action space instead of the PCA and palm position targets. However, with this geometric fabric design, found that training times were very long even for single objects and multi-object training did not succeed at all. For example, Figure 6(right) shows RI training curves for both geometric fabric designs and reveals that the fabric with joint position actions takes four times longer to achieve the same level of grasping performance. Moreover, due to the underspecified nature of the reward function and higher dimensional action space for the espace fabric, awkward and unnatural grasping behaviors emerged. For example, Figure 6(left) shows how the policy tries to grasp objects between the ring and middle finger and the thumb and palm. These unnatural strategies did not transfer well. Moreover, finger deranged finger arrangements also emerged, resulting in aesthetic degradation of the behavior.
14

---

Legend:
Body Point x
Wall
Camera
Robot Body Collision Sphere
Wall
Direction to closest point on environment object, R;
Camera
Wall
Distance from robot body collision sphere obstacle, di
Camera
Smallast d,, obstacle closest to collision
Table
Base acceleration away from collision, X
Lower Table
Figure 5: We use a geometric fabric controller with integrated environment and self-collision avoid- ance. We visualize the geometric fabric's 's collision model, which models the robot as set of spheres and the environment as a set of boxes.
20144 13-3535
10 23
Figure 6: RL training over a cspace fabric resulted in awkward and unnatural strategies emerging (left) like grasping between the ring and middle fingers and deranged finger configurations. Criti- cally, high grasping performance was achieved much faster with RL over DextrAH's chosen fabric versus espace fabric (right) during single object assessment. High grasping performance for multi-object training was not achieved at all with the espace fabric.
D Human Motion Retargeting and Fabric PCA Taskmap
An important aspect for a controller is the action space it creates for overlying modules. Within the context of policy learning, the chosen action space can have significant impacts on learning per- formance and overall behavior. The action space for DextrAH-G enables finger-arm coordination, promotes natural grasping motions, and facilitates RL training for grasping many different objects in simulation scale. We now focus our attention on part of the action space that involves the fin- gers. To create an efficient finger action space for reinforcement learning, find linear taskmap by running PCA on Allegro finger joint motions derived from retargeting human grasping data. To begin, we leverage few demonstrations of grasping data from the DexYCB dataset in [47]. This dataset contains 3D point motion traces of human fingertips, joints, and palm humans throughout object grasping trials. Since the Allegro hand is much bigger than human hand and only has four fingers, the index, middle, ring, and thumb fingertip points, X₁, 12 were scaled (scaling factor 1.6) and aligned with the Allegro hand, x,, 12 (stacked fingertip points). We aligned the points by creating palm- faxe-fixed coordinate system for both the human hand and Allegro hand by similarly placing origin and orthonormal axes in [48]. Points were expressed in their respective
15

---

palm-fixed fixed coordinate system. We then optimize the following loss, L, that solves for Allegro joint angles, 4r- for each data point Xn, (in sequence). For the first datapoint, we initialize q, to zeros. Optimization solves for an unconstrained q. which is then passed through a differentiable saturation function q. - (lanh((q) + l) - q) + q. where q and q are the upper and lower joint positional limits of the hand. The loss we optimize via ADAM
L(q,) = =>||x, axxill 1|2 + (1->>]|x, - xzl2² Al1q, - -reg
(4)
where x [x x x² x stacks x, where X is single 3D point used for encouraging power or precision grasps. For instance, we place the point on the robot palm to encourage the fingers to fully curl, resulting in power grasp. We place the point, centrally located among the fingertips to encourage precision grasping. X, are the stacked Allegro fingertip points by running forward kinematics on qr- = i+1 is a blend factor that shrinks to 0 over a motion data trace (i is the index of the trace and n is the number of datapoints a trace). Retargeting for the first data point in a motion trace places all the weight on optimizing the first term in (4). Retargeting for the last data point in the motion trace places all the weight in optimizing the second term in (4). This effectively shifts the optimization objective from trying to generate hand shape that mimics that in the dataset towards one that drives the fingertips towards power precision grasp. Finally, A E R is regularization weight of the retargeted angles towards some fixed, desired angles, Greg. For precision grip. 9reg = [0, 0, O, O, 0, 0, 0, 0, 0, 0, 0, 0, 1.0, 0.75, 0, 0] (an opposed thumb, fingers straight configuration.. For a power grip, Greg = [0, 1, 1,00, 1, 0.75, 0, 0] (an opposed thumb, fingers curled configuration). Overall, this retargeting process generates a joint- level dataset of Allegro precision and power grasping motions. Please see video for demonstration of retargeted motions.
Given the Allegro grasping motion dataset, we apply PCA to find rectangular projection matrix with the most dominant eigenvectors. After several applications of PCA, we discover that the five most dominant cigenvectors explain 98% the variance the data, which is also consistent with [17]. This rectangular matrix serves as a taskmap within the fabrics framework and exposes a 5- dimensional action space for a fabric-guided policy. Not only did this action space facilitate the learning of perforreff grasping behavior via RI., the grasping behavior itself looked natural as well without any additional reward shaping required. Please see video for fabric behavior that leverages this action space. Figure 7 shows visual comparison of example grasps using PCA action spaces of different sizes.
E Additional Teacher Privileged FGP Details
E.1 Reward Function and Reset Conditions
We define our reward as weighted sum of individual reward terms r Σ, way; E R, where r; E IR and E IR are the reward and weight associated with the i-th reward term, respectively. Let e R be an error term we want to minimize and Csmaltest E IR be the smallest the error term has been in this episode so far. We define a stateful function minimize(e) = max(Esmalleat - e, 0), which only gives a positive reward if the error term drops below the smallest has been so far, otherwise gets no reward. This means that when the reward is positive, has dropped below Esmallest. so we subsequently update esmailest the teacher policy does not get additional reward for staying the same error in subsequent timesteps.
16

---

2D PCA
5D PCA
Figure 7: We visualize example grasps in 2D PCA action space and 5D PCA action space. The 5D PCA action space enables more diverse grasping behavior while staying on the manifold of numan-like grasping motions.
We define (c) = 1 if e is true else 0, z(x) = and lifted(x) 1 (z(xx > zlifted). We use the following reward terms:
Tta-nbi minimize fingertips finggrrtiiiiii Xabil|)
Ttiftt minimize(Zujjen (Zliftedd ((Xat)) (1 (xfted(xobi))
Plifted lifted(Xot,) for the first time
fto-goal minimize([ (I Xobjll) x lifted(Xobr)
Treached [[| ([|x Xobj < dsuccess)
Teuccess 1 [Treached = 1 for T success consecutive timesteps) x (Tmax T)
where Stifted Ztable 0.2m, Ztahte is the posititon of the table, dauccess 0.1m, Tsuccess 15, T Tmaz = 150 is the max episode length, and 7' is the current timestep (with actions being taken at 15 Hz, this is second for success and 10 seconds for max episode length). The associated weights are Wto-obj = 5, Whift = 50, Wrifted 50, Wto-gool = 1, 0000 Wreached 40, Wsucoess 100.
We reset the environment if the object has fallen below the table, if the robot received the success reward Fecccees (robot held the object at the goal position for I success consecutive timesteps), if the episode time limit is reached:
reset aany(z(Xonj) < Ztables I Tsuceess = 1, T Tmaz)
We follow reward structure similar to Petrenko et al. [49]. First, we only have positive rewards, so there is no incentive for the policy to end the episode prematurely to avoid punishment. Second, we use the stateful minimize(e) function to only reward the policy it has reduced the error below the smallest has been before. This ensures that the teacher policy only gets reward for getting closer to target than it ever has this episode, so there no reward for just staying place (unless is at the goal). It is also easier tune reward weights because the maximum cumulative reward from one of these terms is simply the initial value of e at the start of the episode. Third, use lifted lifted term to turn on and off reward terms to prevent the robot from getting goo- rewards by pushing the object along the table without lifting it first.
We tuned the weights in a very simple manner by ensuring that each new reward term contributes more the total accumulated reward than the previous. At initialization, the object roughly 0.5 meters away from the hand, the total reward from rto-obj is about Wio-obó x 0.5 2.5. The object is lifted by 0.2 m, so the total reward from Fistt is about waft 0.2 = 10. The total reward from
17

---

Plifted is 50. The distance from the object to the goal position after being lifted is roughly 0.3 m, so the total reward from Tto- is about Wto-gool x 0.3 300. Treeched is given at every timestep that the object has reached the goal, so this could be at most Wrecked X (Tmax - T) - 40 x 150 = 6000. However, we want to ensure that there is sufficient incentive to acquire Y Tsuccess so that the policy holds the object at the goal position, or else the policy may purposely go into and out of the reached region to prevent early termination of the episode from succeeding. Thus, than Wreached. that the total reward from success Wsuccess (Tmaz - T) 100 150 15000 will always be greater than the total reward from reached Wreached x (Tmax T) v 6000, as long as the robot can succeed. Thus, there will always be incentive and greater potential reward for getting T Tsuccess over just getting Treached- Again, note that the cumulative reward from every reward term larger than its previous term, and there is minimal opportunity for the policy to exploit reward hacking behavior.
Wsuccess higher
F,2 Initial State Distribution
At the beginning of each episode, we sample an object pose and a robot configuration. The object position is uniformly sampled from [-0.18125m, 0.181 (8m12m x [-0.29m, 0.29m] x [0.05m, 0.051m] with respect the center of the table, which is half of the table length and width. For object ori- entation, we note that some objects in their default orientation are upright, so almost every sampled orientation will have the object topple over. Because we want our policy to handle both the upright and not upright case, we set the object orientation to the upright orientation with probability of 0.5 and uniformly randomly sampled orientation with probability 0.5. For the robot's initial config- uration, we set the robot's joint position to a predefined default position (incorporating elbow flare) with uniformly sampled noise with sampling bounds 10% of the joint limit range. We also ran- domly sample small joint velocities in [-0.1, 0.1]. Fabric-basee collision detection is then employed to conduct rejection sampling to ensure that the initial state is not in collision.
E.3 Random Wrench Perturbations
At each timestep, we sample random unit vectors Uy R and u, E R.3 then apply random forces fperturh = f and random torques Tperturb = Iu, with probability = 0.1, where m E is the object mass, I E Rax3 is the object inertia, fecale = 50 is a force scaling parameter, and Tecale 100 a torque scaling parameter.
E.4
Puse Noise
The policy observes Xobj = Xobj + 0x,uncorr + corr and 9ohy = qoh3 + Uqquncorr + q.corr. where uncorr ~ NO, (20,(zys,.noorr) and Alq.unoor NO,, "rpy,ancory are uncorrelated noise sampled once every simulation timestep, and 7xxcorr (O, Ozyz,corr) and nq.cor -N(0, Tryg.corr) correlated noise sampled once at the start of each simulation episode and kept the same for the duration of the episode, with Ozzz,wncorr Ozyz.cor 0.02m and Orpg.uncor = Oppy.cor 0. O.lr Because poses are represented as quaternions, we convert the roll-pitch-yaw noise to a quaternion then perform quaternion multiplication with the ground truth quaternion.
E.5 Domain Randomization
We apply randomization to many parts of the system, including the robot's 's PD gains, gravity, object mass, and object friction, and we apply observation and action noise. The exact parameters are shown in Table 2.
E.6 Teacher Architecture
Our network architecture is shown in Figure The critic uses a Multi-Layer Perceptron (MLP) network [50] because it has access all privileged information, it does not need to capture temporal dependencies. The teacher policy uses MLP layer followed by Long-Short Term
18

---

Group
Parameter
Type
Distribution
Operation
Range
Robot
Mass
Scaling
uniform
scaling
[0.3, 3.0]
Friction
Scaling
uniform
scaling
[0.5, 1.1]
Restitution
Additive
uniform
additive
[0, 0..4
Joint Stiffness
Scaling
loguniform
scaling
[0.5, 2]
Joint Damping
Scaling
loguniform
scaling
[0.3, 3J
Object
Mass
Scaling
uniform
scaling
[0.3, 3]
Friction
Scaling
uniform
scaling
[0.5, 1.1]
Restitution
Additive
uniform
additive
[0, 0.4]
Table
Friction
Scaling
uniform
scaling additive
[0.5, 1.1]
Restitution
Additive
uniform
[0, 0.4]
Observation
Uncorrelated Noise
Gaussian
gaussian
additive additive
[0, 0.005]
Correlated Noise
Gaussian
gaussian
[0.0, 0.01]
Action
Uncorrelated Noise Correlated Noise
Gaussian
gaussian
additive
[0, 0.05]
Gaussian
gaussian
additive
[0.0, 0.02]
Environment
Gravity
Gaussian
gaussian
additive
[0, 0.5]
Table 2: Randomization Parameters
Memory (LSTM) layer [51] capture temporal dependencies in the observations. Because LSTMs have been shown to exhibit less stable training [52], we add skip connection around the LSTM allow the policy pass information around the L.STM. We find this improves the stability and quality of training because the skip connection treats the LSTM outputs residuals, so the stability of the policy training whole is not as sensitive to the LSTM.
E.7 Teacher Observation Details
In this section, we discuss some details about the choice of inputs in the teacher observation. We do not include object velocity because is difficult to accurately estimate from real world sensors. We also do not require object shape information, as we provide a one-hot embedding that specifies the object, so the policy knows which object is trying to grasp. We also provide goal object positions as opposed to goal object poses because occlusion of the object will prevent the control policy from having access to accurate object pose information when the object lifted.
E.8 Object Datasets
We train DextrAH-G on the Visual Dexterity object dataset [45], which consists of 150 diverse objects, including footwear, kitchenware, toys, and more. Chen et al. [45] demonstrate that their point cloud policy can generalize novel object geometries after being trained this dataset. We remove 10 of these objects due to simulation errors, which leaves with 140 objects, shown in Figure 8. We preprocess the object meshes by computing the mesh centroid using Trimesh [53], and then transforming the vertices such that the new mesh centroid is at the origin. We do this so that the object positions given by the simulator would be exactly at the object centroids, which helps with both teacher policy and student policy training. During simulation, we use V-HACD perform convex decomposition perform fast collision detection for these objects. We use the default V-HACD parameters as provided by Isaac Gym [54].
E.9 Modifications to Improve Performance on Low Profile Objects
Low-profile objects can be very difficult to grasp because they require the hand be very close the table or make contact with the table perform the grasp. This causes many grasp planning methods fail because they motion planning algorithms to find collision-free trajectory the pre-grasp pose, which can be challenging or impossible for the grasps of many low-profile objects.
19

---

Figure 8: Dataset used in simulation. Objects enumeration starts top left and ends bottom right: e.g. object 15 would be indexed as [1, 1] Python this was an array data structure. Circles around the object indicate that they were easiest (green) or hardest (red) for the expert policy to grasp. Further information on per-object success analysis is provided in Appendix F.4
Our geometric fabric allows our policy to have light contact with obstacles. However, it can also restrict exploration near these collision surfaces, which can result in slow reinforcement learning training. To overcome this, train teacher policy with curriculum espace damping. Cspace damping is geometric fabric parameter that influences how quickly the robot can move and how much collision contact is allowed.
We start training with espace damping of O, which allows for more environment contact and maximum exploration. From there, we increase the espace damping by 0.1 when the average success rate of the policy goes above a threshold of 90%. We repeat this until espace damping of 10 is reached, which allows minimal contact with the environment and can be safely deployed in the real world.
E.10 Effect of Random Seed on Reinforcement Learning
We find that random seed plays a factor in the behavior of reinforcement learning policies. Different random seeds result in different high-level grasping behavior and different real-world performance. We believe this is because our reinforcement learning formulation is under-constrained problem, as we only give task rewards, so there are many possible policies that could achieve high reward. Additionally, train policy completely from scratch and give shaping rewards, so the domness of exploration to train multiple policies with different random seeds, and then comparing simulation results and visually viewing policy behavior, and testing policies in the real world. This can sometimes result "random seed shootout", where we simply test several policies out in the real world and choose the best one (this strategy is the same as in [52, 13]. These different policies often have different strengths and weaknesses (e.g. some better at low-profile objects, some better at large objects). We have found that simulation performance can give some indication about real-world performance, but there is no replacement for real-world testing.
result in very different learned behaviors. Our approach to this has been
We also find that for a given random seed, the policy typically sticks to the same general strategy over the training run, with slight improvements to handle different objects and adjust to curriculum updates. generally has similar approach to grasping for most objects. We hypothesize that we could enable more diverse grasping behavior by training multiple teacher policies and critics, each supervise the student policy match
with different behavior. Then during policy distillation, the action of any of the teachers or the teacher policy with the highest predicted value from the critic.
20

---

E.11 Training and Simulation Details
We train our policy using Isaac Gym [54], a high perfforrrrmmece simulator that leverages GPU par- allelization, which allows us simulate 8192 robots simultaneously per GPU. We train using four NVIDIA V100 GPUs, each with 32 GB of VRAM. This compute allows us to run simulation at about 20k FPS, where each frame is one action step with a control timestep of 66.7 ms (15 Hz) and is broken up into 4 simulation timesteps of 16.7 ms (60 Hz). We train for about 4.7B frames (about 9000 policy updates), which takes about 68 hours (wall clock time). This amounts about 10 years of simulated training time (4.7B/ 15 / 3600 / 24 / 365). We train our teacher policy using Proximal Policy Optimization (PPO) [55] using a highly-optimizee GPU implementation called rl games [56], which uses vectorized observations and actions for faster training.
We train our teacher policy with a learning rate of 5e-4, a discount factor of 0.998, entropy coeffi- cient of 0, and PPO clipping interval Eclip of 0.2. We also normalize the observations, values, and advantages, and we train the policy with 5 mini-epochs per policy update. Using a horizon length of 16 (number of timesteps between updates for each robot, with all robots running in parallel), and 8192 simulated robots. We use critic MLP of size [512, 512, 256, 128), policy MLP of size [512, 512), and a policy LSTM of size 1024.
E.12 Privileged FGP Training Curves
As seen in Figure 9, the privileged FGP trains well within 20 hours of wall-clock time with the vast majority of objects successfully being lifted from the table, brought to target position, and held there. The remaining training time residually improved policy reward with correspondingly residual improvements in policy performance across all scoped metrics. Overall, the required training time for this complex task is favorable and in line with other related manipulation works [52, 13].
E.13 Comparisons of Grasps
To encourage policy robustness to noise, disturbances, and discrepancies in physics, we apply ran- dom force-torque perturbations to the object during grasping, lower contact friction, object pose noise, and significant domain randomization. Like prior works [52, 13], these additions are neces- sary for strong sim2real transfer for robot manipulation problems. Without it, unnatural and frail strategies emerge as seen Figures 10, 11, where policies attempt pick up objects without proper force- and form-closed fingers arrangements. Attempts to grasp objects between the middle and ring fingers only between the index and ring fingers only were observed. With the above ad- ditions, policy strategies become much more robust as seen in Figures 12 and ultimately facilitated DextrAH-G's high real-world performance.
F Additional Student Depth FGP Details
F.1
Depth Image Details
We simulation-rendered depth images are entirely free of noise as shown in Figure 13. To address the visual sim2real gap, we add a variety of augmentations the rendered depth image. Specifi- cally, the following augmentations
added: 1) pixel value would be set 0 with probability Peropout 0.003, 2) pixel value would be set to random value E (-0.5, -1.3) with probability Prondu = 0.003, 3) a linear segment of pixels of up to 18 pixels length and 3 pixels width would appear with probability of Pstick = 0.0025 to mimic robot wires and other artifacts, and 4) the un- correlated and correlated depth noise models and their parameters are used exactly as reported in [57]. We also stochastically perturbed the depth camera placement in simulation during distillation so that the distilled FGP gained robustness to calibration errors in the real world. The effect of these augmentations can be seen in Figure 13. See video to better visualize the depth image noise in both simulation and the real-world
21

---

metrics/manjltteed
oegin
0.5
0.5
04
6.2
0.2
Time
06
Tiew Pourt
Tope Boac
metrikt/mmat/zuun propuuudddddttttttto metricc
Figure 9: We visualize various metrics of reinforcement learning training performance over wall clock time. The total training time is about 68 hours. To compute these metrics, we keep a running average over the 100 most recently finished episodes. "lifted" is boolean that is 1 if the object has been lifted at least 20 cm above the table surface during the episode. "reached success region" is boolean that is 1 if the object has been less than 10 cm away from the goal position during the episode. "has_end caass_nough_comme consecutive_uucceseeeeeooo
_ettttoeedddd__eeeeeeee is a boolean that is if the object has been held within 10 from the goal position for 1 simulated second (15 consecutive timesteps). "reward" is the total accumulated reward at the end of the episode. "num-drops. dmtrrceertrricc metricc is the number of times
that the object has been lifted above 20 em and then subsequently brought below 10 em (typically when the object has been dropped).
Unstable grasps emerge with naive environment implementation without random force and torque perturbations, lower friction, object pose noise, and domain randomization. The policy able to find a grasping strategy that works very reliably in simulation, but fails in the real world due to real world perception and control errors.
Figure 10:
22

---

Unstable grasps emerge with naive environment implementation without random force and torque perturbations, lower friction, object pose noise, and domain randomization. The policy is able to find a grasping strategy that works very reliably in simulation, but fails in the real world due to real world perception and control errors.
Figure 11:
Robust grasps emerge when we add in random force and torque perturbations, lower friction, object pose noise, and domain randomization. These force the policy to learn grasping strategies that are robust and demonstrate retry behavior.
Figure 12:
F.2 Distillation
During distillation, we run = 6 steps before running backpropagation through all steps (BPTT (58)) and updating deepth weights. We helieve that a similar effect could have been achieved by using the last n steps (concatenated) as input to stateless model. This number steps corresponds time window of 0.4s. Additionally, we update weights every time an environment is done the object falls off the table, 10 seconds expires, the robot grasps the object.
Typically, the time needed for the student to learn to match the teacher is about 12 hours (wall clock), or 140 rollouts across 480 parallel environments on a single NVIDIA GeForce RTX 3090. This compute allows us to run simulation at about 140 FPS, where each frame one action step with control timestep 66.7 ms (15 Hz) and is broken up into simulation timesteps of 16.7 ms (60 Hz). This amounts to about 6.05M frames, which about 4.67 simulated days (6.05M 15 / 3600 24).
F.3 Student Architecture
The student architecture consists of simple encoders for different modalities and state-based net- work that processes encodings. Our architecture is shown in Figure 2. Orobot and Xgoal encoded
23

---

Figure 13: Left: clean depth image from the simulator. Middle: depth image from the simulator after added noise. Right: depth image in the real world (different object from the left two images).
using 3-layer MLPs (512,256,128) with elu activation, while I is encoded using three convolutional layers with increasing depth (16,32,64), kernel size 3, stride 1, padding 1, followed by max- operations (kernel size 2, stride 2) with ReLU activations, and followed by a 2-layer MLP (128, 128) also with ReLU activation. Encodings of all modalities are concatenated and passed through GRU layer (1 hidden layer, 1024 units) that predicts å. The whole architecture is learned from scratch. We opt for this rather simple architecture due to GPU memory constraints since the whole distillation single GPU (RTX 3090). We believe the student architecture would benefit
process is executed from a better perception backbone (e.g. ResNet).
We designed the student Tedepth to be a state-based model (GRU) because the teacher Tteacher also state-based model (LSTM),, and we found that a state-based teacher achieves higher success rates than stateless teacher (MLP). Further, object's 's position during occlusion since the model has seen the object before the robot got close to the object to occlude it. This means any model architecture that takes history in some way in theory has the ability required to mimic the expert. We opted for GRU, but we believe any form of a state-based model (RNN, LSTM, GRU) or a stateless model (MLP) with history as concatenated input should work.
believe having state allows ddepth reason about the
E.4 Simulation Experiments
We evaluate our teacher and student policy in the simulator across 140 different objects from the training dataset. The objects are visualized in Figure The policy has a 10 second window to grasp the object. If the object lifted the target position in the air, the attempt is labeled as successful. Otherwise, it's failed attempt. Results are shown in Figure 14.
Since the teacher policy has access to privileged information, it achieves a slightly higher success rate compared to the student. The average time needed to grasp an object about 4 seconds, which matches our real-world observations. We believe this type of per-object analysis could be leveraged during training to explicitly train more on the objects that are harder to grasp.
G DextrAH-G State Machine for Bin Packing
To complete bin-packing application, we leverage DextrAH-G and simple state machine. The bin packing program initializes by immediately engaging DextrAH-G. If the z-coordinate (height off the table) of the predicted object position is sufficiently high, then we freeze the last issued PCA action and give the fabric a goal pose over the bin for a fixed amount of time. Afterwards, a finger-opening PCA command issued to the fabric to release the object for a fixed amount of time. Finally, a nominal PCA and pose target are issued to the fabric for fixed amount of time to bring the robot back to a nominal pose. DextrAH-G's high performance and auxiliary object position prediction makes application programming extremely simple. The only additional complexity is that if the palm height ever swings too high (indicating policy collapse), then DextrAH-G is deactivated and
24

---

Aaadyii for diject
Cxpat: 18012.05
Figure 14: Success rates per object (10s allowed per attempt). Figure 8 visualizes the objects used
issue the geometric fabric a single command to bring the robot back nominal configuration for fixed amount of time. Afterwards, re-engage DextrAH-G.
H Nomenclature
We summarize our nomenclature in Table 3.
I Additional Real Experiment Analysis
We create several additional plots to further capture DextrAH-G's performance with greater granu- larity. Figure 15 characterizes DextrAH-G's success and failure modes. Overall, DextrAH-G has 89% grasp success rate and an 87% transport success rate across the many tested objects. The slight drop between these two covers cases where the object falls out of the hand during transportation. The principal failure mode for grasping is the robot pushing the object out of the allowable work area as delincated by the geometric fabric. The less frequent failure mode occurs when DextrAH-G repeatedly attempts to grasp an object, but consistently fails. In this case, the object reset by a human.
A time-series plot of CS is shown in Figure 16. Overall, DextrAH-G consistently grasps and transports several objects in row before experiencing failure. Several times, DextrAH-G moved more than 20 objects in a row before experiencing a failure. DextrAH-G did produce failures within 5 CS several times due to the previously mentioned failures modes.
Figures 17 and 18 report DextrAH-G's speed performance index. Overall, DextrAH-G successfully grasps and transports 5.64 objects per minute and DextrAH-G handles 42% of objects within 7.3 s. With persistent uptime, dexterous grasping with this speed level becomes useful for real-world applications.
Figure 19 shows the success rates per object for all tested objects in the continuous run trials. Over- all, most objects were handled well, but few like the pot, small bottle, and green cup caused greater level of handling errors. Training DextrAH-G on even greater variety of objects in simulation could help close the handling gap these objects and further improve DextrAH-G's performance more broadly.
25

---

3.1 Geometric Fabrics and Fabric-Guided Policies (FGPs)
Symbol
Meaning
My Rexn
Positive-dcfinite system metric (mass), which captures system prioritization.
f E Ra
Noriial path generating geometric force.
(a) E R"
Additional driving force of some action E R".
9/.9/. Gr E R"
Position, velocity, and acceleration of the fabric (9) used PD target control robol).
x = (fx(q) E R
Origin of each sphere used to model the robot geometry (computed through forward kinematics).
F, E R
Closest point on collision body i to a given robot body sphere.
n; R
Direction from given robot body sphere the closest point collision body i.
di E
Signed distance between a given robot body sphere and collision body
de E R+
Lower-bounde distance between a given robot body sphere and collision body s.
X1, R3
Bave acceleration response per robol budy sphere away from collision.
M2 E R
Base metric response per robot body sphere for collision avoidance.
8/ E R
Smooth velocity gale that goes high when this robot body sphere is moving lowards collision body s.
U; E
Signed impact speed that is negative when moving towards collision body
A E R5X16
First live components from PCA on grasping motion data retargeted from human hand to Allegro hand.
f.target E R23
Target palm position.
fftorget E g
Target palm orientation in Euler angles. Target PCA position for the fingers.
Xpca.target E R
3.2 Teacher Privileged FGP Training (Reinforcement Learning)
Symbol
Meaning
Oprivileged
A subset of privileged state information provided to the teacher policy.
"privileged Oprivseeedd
Teacher policy that is truined with Oprivsteged- All privileged state information provided to the critic.
((s)
Critic value function that is trained with s.
vyohot
Rohot state information.
Xyool C R
Goal object position.
Opbj
Object state information provided to the teacher policy.
Xoby E R23
Noisy object position.
Gotj C R²
Noisy object quatemnio.
(0, 1) Notjects
Object one-hot embedding.
Xpalm: Xpalm- a Xpalm-vt R3
Positions of three points the palm.
X Xfingertips C R RNgingara X3
Positions of the fingertips.
Sprivdeged E R
Privileged state information provided only the critic.
faof C CRA
Robot joint forces.
fiiners E Jingers Rd
Fingertip contact forces.
Xobj R
True object position.
4oby E R4
Truc object quatemion.
Yobi C RJ
True object velocity.
Wohj C R
True angular velocity.
a E R
Policy action, which passed to the underlying geometrie fabric.
Lppo C R
PPO loss.
3.3 Student Depth FGP Training (Policy Distillation)
Symbol
Meaning
Orlepth
Observation provided to the student policy.
Teepph (Odepth)
Student policy that is trained with ddepth-
1 (0.5, 1.3] 1.3]10x
Raw depth image.
à E RII
Student predicted actions.
Xobj R3
Predicted object position.
L R
Student supervision loss.
Laction E R
Student action loss.
Lpne R
Student position loss.
B R
Weight student position loss.
Table 3: Nomenclature used in Section 3. Symbols used in Figure 2 are highlighted in yellow.
26

---

DextrAH-G Grasp-and-Trande Performance Analysis
Transport Success: 223 (8746)
All Attempts: 258 (100%)
Grasp Success: 228 (89%)
Grasp: Grasp Placemenit (196)
PaalspporFaalleeeet Rushed out Bounds: 20 (8%6)
Grasp Failure: 20 (11%)
Repeated Grasp Missessi 8 (396)
Figure 15: Sankey diagram showing how often DextrAH-G succeeds, how often it fails, and how the failures occurred. While DextrAH-G achieves 87% success rate unseen test objects, that the leading causes failure are from accidental pushes that push the object out of the graspable region (8%) and repeated grasp misses for more challenging objects (3%). The other failure modes were a loose grip resulting in the object being dropped before reaching the bin (1%) and poor grasp placement resulting the object being dropped before reaching the bin (1%). We define "Grasp Success" as the robot's s actions resulting in the object being grasped and lifted off the table more than inch. We define "Transport Success" as the robot's 's actions resulting in the object grasped and lifted off the table, and then placed into the bin.
In tandem with the results in Figure find that the primary factors that affect policy performance include object size (small bottle), object slipperiness (green cup), object's propensity to easily roll (apple), transparent objects (sanitizer bottle), and geometric aspects of the object that increase the likelihood of the fingers "catching" or "snagging" on the object.
Finally, we scope DextrAH-G's signals over short horizon during grasping and plot the results in Figure 20. As seen, the depth FGP can output quickly shifting actions while the geometric fabric produces smooth joint angle targets. Moreover, since the desired velocity is set to zero for the PD controller, the arm's physical position lags behind the targets by about 0.2 The hand's physical position lags behind the targets by about 0.1 Despite this phase lag, DextrAH-G operates well due to RL training for these lagged dynamics. We also see increased separation between desired and measured joint angle positions during the later phase of DextrAH-G exccution because the robot is grasping the object. The increased tracking error induces contact forces between the robot and the object which facilitates grasping. Overall, the hand experiences greater joint angle swings during execution than the This quality desirable and indicates that the hand is doing much of the mechanical work required for grasping while minimally moving the arm. It also the case that arm joint angle movements translate to much greater Cartesian palm movement so only minimal arm angular movement is required for effectively guiding the hand during grasping.
27

---

Consecutive Successful Srnng--nddTTrrrspottt Over Contiruouu Run
23
Contenf Sirresthl Cow. wow.--n--hhooooe tnohhroottt
20
15
10
500
1000
1500
2000
Time (6)
Figure 16:
From aggregating 256 attempted grasp-and across 30 objects, highest number consecutive successes was 27.
DextrAH Transport
5,80 succsnofd quapendkosnspriamn
200
150
Cambebbo Com
100
50
20 DOToeeann
Time
Figure 17: From aggregating 256 attempted grasp-and- -rannndstttootsssssss across 30 objects, we demonstrate 5.64 successful grasp-and-transports per minute, with 0.53 interventions per minute and 0.30 failures per minute. We define successful object being grasped and lifted off the table, and then dropped into the bin. We define interventions as the human adjusting the position or orientation of the object after the robot has failed to grasp the object. We define failures as the robot taking actions that result in the object falling off the table or moving far away enough that human decides to move onto another object instead of intervening. If the robot misses grasp or drops the object, but then subsequently retries successfully without human intervention, this counts as one successful grasp-and-tee -nd-transport, zero interventions, and zero failures. Note that these rates aggregate results of total cumulative count divided by total time, so this rate successful trasp-ann-transports is affected by the time taken for interventions and failures.
as the robot taking actions that result in the
28

---

Distribution Grasp-are cicippand--raaaaen Transport
100
90
80
70
Froqurcy
20
10
15.0
12.5
32.5
30.0
boteoon Successtul Tanoports
Figure 18: We define grasp-and-transpee -ransport time as the time between successful grasp-and-transae -ra-trassports. Note that with this method of computing times, if there is intervention or failure between two successful grasp-and-transports. this results in a longer grasp-and-transpon time, which makes these conservative estimates of grasp-and-transoor time. We find the median grasp-and-trange anndtrranpoort time be 8 seconds. Roughly 42% of grasp-and ann-transportt took between 6.0 7.3 seconds.
Par-Otject Results from Continucus Runs
Successtu Graep and Transoorts
interventions
Fellures
bawch wetl oren
24
Figure 19: We visualize real-world per-object results over continuous runs across objects. DextrAH-G performed best on real-world objects such the brick, spam, duck, mouse box, and container. It performed least well on real-world objects such as the small bottle, pot, and green cup. We define successful mousp-nnd-transpee grasped and lifted off the table, and then dropped into the bin. We define intervention as the human adjusting the position orientation of the object after the robot has failed to grasp the object. We define failure as the robot taking actions that result in the object falling off the table or moving far away enough that the human decides to robot misses grasp or drops the object, but then subsequently retries successfully without human intervention, this counts one successful grasp-and-transport, zero interventions, and failures.
the robot taking actions that result in the object being
onto another object instead of intervening. If the
29

---

30
155
7.5
125
2.-
1.09
12
025
0.3
060
0.0
2.0
1.0
Tirm
Time
10-
-3.
-15-
-20
Figure 20: System signals during DextrAH-G deployment. The depth FGP outputs hand PCA and arm pose actions (left side). The geometric fabric produces joint angle targets (right side, solid line). The joint PD controller forces the real robot to track these joint angle targets resulting in correlated measured motion (right side, dotted line).
30


Reviews:
[{"review_id": "9MXtPLdKF3", "reviewer": "Reviewer_LfmX", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposes to use teacher-student distillation with a geometric fabric controller for dexterous grasping. They train the controller in simulation and transfer it to the real-world.", "summary_of_recommendation": "I would recommend acceptance for now due to the strong hardware results. However, I want to emphasize that several experiments and clarifications are needed during the revision period.", "technical_quality": 4, "clarity_of_presentation": 3, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "About object position prediction: What is the system's performance with and without position prediction?\n\nAre there alternative ways to do collision avoidance? For example, one could add a penalty term in reinforcement learning training to penalize self-collision and the distance between the object and the environment.\n\nCan the proposed controller improve the performance or safety of a worse policy?\n\nWhat is the single grasp success rate for the policy?\n\nIs there an evaluation of the relationship between performance and object scale?\n\nIt would be helpful to evaluate the student policy with thousands of parallel environments. One can achieve this using multiple GPU parallelization.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DEKlziVSCj", "reviewer": "Reviewer_1JfV", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper introduces a novel depth-based dexterous grasping policy trained entirely in simulation. The proposed method, DextrAH-G, combines reinforcement learning, geometric fabrics, and teacher-student distillation to enable a 23-motor arm-hand robot to safely and continuously grasp and transport a variety of objects.", "summary_of_recommendation": "The paper proposes a simple but effective method for dexterous grasping, although showing relatively simple grasping behavior.", "technical_quality": 3, "clarity_of_presentation": 4, "potential_impact": 3, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "I would like the authors to answer the question in the weakness part.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zwUmxxtPoP", "reviewer": "Reviewer_7Akv", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 5, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper presents an RL approach to learn closed-loop, robust visuomotor grasping policies through a combination of underlying geometric fabric controller and teacher-student distillation policy.", "summary_of_recommendation": "See my review.", "technical_quality": 4, "clarity_of_presentation": 3, "potential_impact": 4, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "1. Is there any concrete interpretation of the 5 PCA dimensions of fingers? \n2. If authors need to extend this work to other types of hand-object interaction, is the PCA approach sufficient? Or should we use the full-hand finger joints?\n3. Address the concerns about paper writing in the Review section above, especially revising figure2.\n4. Does the location of the point clouds need to be transformed to a fixed workspace assuming access to a calibrated camera extrinsic? Or it", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes to use teacher-student distillation with a geometric fabric controller for dexterous grasping. It trains the controller in simulation and transfer it to the real-world. Reviewers appreciated the proposed method and experimental design. Reviewers raised concerns about relationships with past work, some claims that are without empirical evidence, some missing experiments, and clarity of presentation in certain sections and figures. The author response addressed reviewer concerns and all reviewers continue to vote for the acceptance of the paper.
Decision: Accept