Original: DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Video
Revised: DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Video

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AfﬁliationAddress email
Revised: AfﬁliationAddress email

Original: Abstract: Dexterous multi-ﬁngered robotic hands have a formidable action space,yet their morphological similarity to the human hand holds immense potential toaccelerate robot learning.
Revised: Abstract: Dexterous multi-ﬁngered robotic hands have a formidable action space,yet their morphological similarity to the human hand holds immense potential toaccelerate robot learning.

Original: We propose DexVIP, an approach to learn dexterousrobotic grasping from human-object interactions present in in-the-wild YouTubevideos.
Revised: We propose DexVIP, an approach to learn dexterousrobotic grasping from human-object interactions present in in-the-wild YouTubevideos.

Original: We do this by curating grasp images from human-object interaction videosand imposing a prior over the agent’s hand pose when learning to grasp with deepreinforcement learning.
Revised: We do this by curating grasp images from human-object interaction videosand imposing a prior over the agent’s hand pose when learning to grasp with deepreinforcement learning.

Original: A key advantage of our method is that the learned policyis able to leverage free-form in-the-wild visual data.
Revised: A key advantage of our method is that the learned policyis able to leverage free-form in-the-wild visual data.

Original: As a result, it can easilyscale to new objects, and it sidesteps the standard practice of collecting humandemonstrations in a lab—a much more expensive and indirect way to capturehuman expertise.
Revised: As a result, it can easilyscale to new objects, and it sidesteps the standard practice of collecting humandemonstrations in a lab—a much more expensive and indirect way to capturehuman expertise.

Original: Through experiments on 27 objects with a 30-DoF simulatedrobot hand, we demonstrate that DexVIP compares favorably to existing approachesthat lack a hand pose prior or rely on specialized tele-operation equipment to obtainhuman demonstrations, while also being faster to train.
Revised: Through experiments on 27 objects with a 30-DoF simulatedrobot hand, we demonstrate that DexVIP compares favorably to existing approachesthat lack a hand pose prior or rely on specialized tele-operation equipment to obtainhuman demonstrations, while also being faster to train.

Original:  Introduction
Revised:  Introduction

Original: Many objects in everyday environments are made for human hands.
Revised: Many objects in everyday environments are made for human hands.

Original: Mugs have handles to grasp;the stove has dials to push and turn; a needle has a small hole through which to weave a tiny thread.
Revised: Mugs have handles to grasp;the stove has dials to push and turn; a needle has a small hole through which to weave a tiny thread.

Original: In order for robots to assist people in human-centric environments, and in order for them to reachnew levels of adept manipulation skill, multi-ﬁngered dexterous robot hands are of great interest asa physical embodiment [1, 2, 3, 4, 5, 6, 7].
Revised: In order for robots to assist people in human-centric environments, and in order for them to reachnew levels of adept manipulation skill, multi-ﬁngered dexterous robot hands are of great interest asa physical embodiment [1, 2, 3, 4, 5, 6, 7].

Original: Unlike common end effectors like parallel jaw grippersor suction cups, a dexterous hand has the potential to execute complex behaviors beyond pushing,pulling and picking and to grasp objects with complex geometries in functionally useful ways [8, 9].
Revised: Unlike common end effectors like parallel jaw grippersor suction cups, a dexterous hand has the potential to execute complex behaviors beyond pushing,pulling and picking and to grasp objects with complex geometries in functionally useful ways [8, 9].

Original: The ﬂexibility of a dexterous robotic hand, however, comes with signiﬁcant learning challenges.
Revised: The ﬂexibility of a dexterous robotic hand, however, comes with signiﬁcant learning challenges.

Original: With articulated joints offering 24 to 30 degrees of freedom (DoF), the action space is formidable.
Revised: With articulated joints offering 24 to 30 degrees of freedom (DoF), the action space is formidable.

Original: At the same time, interacting with new objects having unfamiliar shapes demands a high level ofgeneralization.
Revised: At the same time, interacting with new objects having unfamiliar shapes demands a high level ofgeneralization.

Original: Both factors have prompted exciting research in deep reinforcement learning (RL),where an agent dynamically updates its manipulation strategy using closed-loop feedback controlwith visual sensing while attempting interactions with different objects [2, 5, 7].
Revised: Both factors have prompted exciting research in deep reinforcement learning (RL),where an agent dynamically updates its manipulation strategy using closed-loop feedback controlwith visual sensing while attempting interactions with different objects [2, 5, 7].

Original: To mitigate sample complexity—since many exploratory hand pose trajectories will yield no reward—current methods often incorporate imitation learning [1, 2, 3, 4, 10].
Revised: To mitigate sample complexity—since many exploratory hand pose trajectories will yield no reward—current methods often incorporate imitation learning [1, 2, 3, 4, 10].

Original: With imitation, expert (human)demonstrations provided by teleoperation in virtual reality [11, 12], mocap [1, 13], or kinestheticmanipulation of the robot’s body [14, 15] are used to steer the RL agent towards desirable state-actionsequences.
Revised: With imitation, expert (human)demonstrations provided by teleoperation in virtual reality [11, 12], mocap [1, 13], or kinestheticmanipulation of the robot’s body [14, 15] are used to steer the RL agent towards desirable state-actionsequences.

Original: Such demonstrations can noticeably accelerate robot learning.
Revised: Such demonstrations can noticeably accelerate robot learning.

Original: However, the existing paradigms for human demonstrations have inherent shortcomings.
Revised: However, the existing paradigms for human demonstrations have inherent shortcomings.

Original: First, theyrequire some degree of specialized setup: a motion glove for the human demonstrator to wear, avirtual reality platform matched to the target robot, a high precision hand and arm tracker, and/orphysical access to the robot equipment itself.
Revised: First, theyrequire some degree of specialized setup: a motion glove for the human demonstrator to wear, avirtual reality platform matched to the target robot, a high precision hand and arm tracker, and/orphysical access to the robot equipment itself.

Original: This in turn restricts demonstrations to lab environments,assumes certain expertise and resources, and entails repeated overhead to add new manipulations
Revised: This in turn restricts demonstrations to lab environments,assumes certain expertise and resources, and entails repeated overhead to add new manipulations

Original: [Figure]
Revised: [Figure]

Original: Imitate
Revised: Imitate

Original: Watch
Revised: Watch

Original: Act or objects.
Revised: Act or objects.

Original: Second, there is an explicit layer of indirection: in conventional methods, a persondoes not do a task with their own hands, but instead enacts a proxy under the constraints of thehardware/software used to collect the demonstration.
Revised: Second, there is an explicit layer of indirection: in conventional methods, a persondoes not do a task with their own hands, but instead enacts a proxy under the constraints of thehardware/software used to collect the demonstration.

Original: For example, in VR, the person needs towatch a screen to judge their success in manipulating a simulated hand and may receive limitedor no force feedback [2, 16].
Revised: For example, in VR, the person needs towatch a screen to judge their success in manipulating a simulated hand and may receive limitedor no force feedback [2, 16].

Original: Similarly, advanced visual teleoperation systems that observe thebare hand [13] still separate the person’s hand from the real-world object being manipulated.
Revised: Similarly, advanced visual teleoperation systems that observe thebare hand [13] still separate the person’s hand from the real-world object being manipulated.

Original: In akinesthetic demonstration, the human expert is furthest removed, manually guiding the robot’s endeffector, which has a very different embodiment [15, 17].
Revised: In akinesthetic demonstration, the human expert is furthest removed, manually guiding the robot’s endeffector, which has a very different embodiment [15, 17].

Original: In light of these challenges, we propose to learn dexterous robot grasping policies by watching peopleinteract with objects in video (see Fig. 1).
Revised: In light of these challenges, we propose to learn dexterous robot grasping policies by watching peopleinteract with objects in video (see Fig. 1).

Original: The main idea is to observe people’s hand poses as they useobjects in the real world in order to establish a 3D hand pose prior that a robot might attempt to matchduring functional grasping.
Revised: The main idea is to observe people’s hand poses as they useobjects in the real world in order to establish a 3D hand pose prior that a robot might attempt to matchduring functional grasping.

Original: Rather than enlist special purpose demonstrations (e.g., record videos atlab tabletops), we turn to in-the-wild Internet videos as the source of the visual prior.
Revised: Rather than enlist special purpose demonstrations (e.g., record videos atlab tabletops), we turn to in-the-wild Internet videos as the source of the visual prior.

Original: Our methodautomatically extracts human hand poses from video frames using a state-of-the-art computer visiontechnique.
Revised: Our methodautomatically extracts human hand poses from video frames using a state-of-the-art computer visiontechnique.

Original: We then deﬁne a deep reinforcement learning model that augments a grasp success rewardwith a reward favoring human-like hand poses upon object contact, while also preferring to grasp theobject around affordance regions predicted from an image-based model.
Revised: We then deﬁne a deep reinforcement learning model that augments a grasp success rewardwith a reward favoring human-like hand poses upon object contact, while also preferring to grasp theobject around affordance regions predicted from an image-based model.

Original: In short, by watching videoof people performing everyday activities with a variety of objects, our agents learn how to approachobjects effectively using their own multi-ﬁngered hand, while also accelerating training.
Revised: In short, by watching videoof people performing everyday activities with a variety of objects, our agents learn how to approachobjects effectively using their own multi-ﬁngered hand, while also accelerating training.

Original: Aside from providing accurate policies and faster learning, our approach has several conceptualadvantages.
Revised: Aside from providing accurate policies and faster learning, our approach has several conceptualadvantages.

Original: First, it removes the indirection discussed above; there is no awkwardness or artiﬁcialityof VR, kinesthetic manipulations, etc.—people in the videos are simply interacting with objects inthe context of their real activity.
Revised: First, it removes the indirection discussed above; there is no awkwardness or artiﬁcialityof VR, kinesthetic manipulations, etc.—people in the videos are simply interacting with objects inthe context of their real activity.

Original: This also promotes learning functional grasps—those that preparethe object for subsequent use—as opposed to grasps that simply lift an object in an arbitrary manner.
Revised: This also promotes learning functional grasps—those that preparethe object for subsequent use—as opposed to grasps that simply lift an object in an arbitrary manner.

Original: New demonstrations are also easy to curate whenever new objects become of interest, since it is amatter of downloading additional video.
Revised: New demonstrations are also easy to curate whenever new objects become of interest, since it is amatter of downloading additional video.

Original: In addition, because our visual model focuses on 3D handpose, it tolerates viewpoint variation and is agnostic to the complex visual surroundings in trainingdata (e.g., variable kitchens, clutter, etc.).
Revised: In addition, because our visual model focuses on 3D handpose, it tolerates viewpoint variation and is agnostic to the complex visual surroundings in trainingdata (e.g., variable kitchens, clutter, etc.).

Original: Finally, the proposed method requires only visual inputfrom the human data—no state-action sequences.
Revised: Finally, the proposed method requires only visual inputfrom the human data—no state-action sequences.

Original: We demonstrate our approach trained with frames from YouTube how-to videos to learn grasps for a30-DoF robot in simulation for a variety of 27 objects.
Revised: We demonstrate our approach trained with frames from YouTube how-to videos to learn grasps for a30-DoF robot in simulation for a variety of 27 objects.

Original: The resulting policy outperforms state-of-theart methods for learning from demonstration and visual affordances [2, 18], while also being 20%more efﬁcient to train.
Revised: The resulting policy outperforms state-of-theart methods for learning from demonstration and visual affordances [2, 18], while also being 20%more efﬁcient to train.

Original: The learned behavior resembles that of natural human-object interactions andoffers an encouraging step in the direction of robot learning from in-the-wild Internet data.
Revised: The learned behavior resembles that of natural human-object interactions andoffers an encouraging step in the direction of robot learning from in-the-wild Internet data.

Original:  Related Work
Revised:  Related Work

Original: Learning to grasp objects Early grasping work explicitly reasons about an object’s 3D shape againstthe gripper [19, 20], whereas learning methods often estimate an object or hand pose followedby model-based planning [21, 8], optionally using supervised learning on visual inputs to predictsuccessful grasps [21, 22, 23, 24, 25].
Revised: Learning to grasp objects Early grasping work explicitly reasons about an object’s 3D shape againstthe gripper [19, 20], whereas learning methods often estimate an object or hand pose followedby model-based planning [21, 8], optionally using supervised learning on visual inputs to predictsuccessful grasps [21, 22, 23, 24, 25].

Original: Most planning methods cater to simple non-dexterous endeffectors like parallel jaw grippers or suction cups that make a control policy easier to codify butneed not yield functional grasps.
Revised: Most planning methods cater to simple non-dexterous endeffectors like parallel jaw grippers or suction cups that make a control policy easier to codify butneed not yield functional grasps.

Original: Rather than plan motions to achieve a grasp, reinforcement learning(RL) methods act in a closed loop with sensing, which has the advantage of dynamically adjusting toobject conditions [26, 27, 28].
Revised: Rather than plan motions to achieve a grasp, reinforcement learning(RL) methods act in a closed loop with sensing, which has the advantage of dynamically adjusting toobject conditions [26, 27, 28].

Original: Only limited work explores RL for dexterous manipulation [5, 7, 18].
Revised: Only limited work explores RL for dexterous manipulation [5, 7, 18].

Original: Our work addresses functional dexterous grasping with RL, but unlike the existing methods it primesagent behavior according to videos of people.
Revised: Our work addresses functional dexterous grasping with RL, but unlike the existing methods it primesagent behavior according to videos of people.

Original: Imitation and learning from demonstration To improve sample complexity, imitation learningfrom expert demonstrations is often used, whether for non-dexterous [29, 30, 31, 32, 33, 15] ordexterous [1, 2, 3, 4] end effectors.
Revised: Imitation and learning from demonstration To improve sample complexity, imitation learningfrom expert demonstrations is often used, whether for non-dexterous [29, 30, 31, 32, 33, 15] ordexterous [1, 2, 3, 4] end effectors.

Original: Researchers often use demonstrations to explore dexterousmanipulation in simulation [2, 3], and recent work shows the promise of sim2real transfer [11, 7].
Revised: Researchers often use demonstrations to explore dexterousmanipulation in simulation [2, 3], and recent work shows the promise of sim2real transfer [11, 7].

Original: Like learning from demonstrations (LfD), our approach aims to learn from human experts, but unliketraditional LfD, our method does so without full state-action trajectories, relying instead only on avisual prior for “good" hand states.
Revised: Like learning from demonstrations (LfD), our approach aims to learn from human experts, but unliketraditional LfD, our method does so without full state-action trajectories, relying instead only on avisual prior for “good" hand states.

Original: Furthermore, our use of in-the-wild video as the source of humanbehavior is new and has all the advantages discussed above.
Revised: Furthermore, our use of in-the-wild video as the source of humanbehavior is new and has all the advantages discussed above.

Original: Imitating visual observations Learning to imitate observations [34] relaxes the requirement ofcapturing state sequences in demonstrations.
Revised: Imitating visual observations Learning to imitate observations [34] relaxes the requirement ofcapturing state sequences in demonstrations.

Original: This line of work includes ideas for overcomingviewpoint differences between ﬁrst and third person visual data [33, 30, 31], multi-task datasetsto learn correspondences between video of people and kinesthetic trajectories of a robot arm [15],few-shot learning [29, 32], and shaping reward functions with video [33, 35, 15, 31].
Revised: This line of work includes ideas for overcomingviewpoint differences between ﬁrst and third person visual data [33, 30, 31], multi-task datasetsto learn correspondences between video of people and kinesthetic trajectories of a robot arm [15],few-shot learning [29, 32], and shaping reward functions with video [33, 35, 15, 31].

Original: However, noneof the prior work uses in-the-wild video to learn dexterous grasping as we propose.
Revised: However, noneof the prior work uses in-the-wild video to learn dexterous grasping as we propose.

Original: By connectingreal video of human-object interactions to a dexterous robotic hand, our method capitalizes onboth the naturalness of the demonstrations as well as the near-shared embodiment.
Revised: By connectingreal video of human-object interactions to a dexterous robotic hand, our method capitalizes onboth the naturalness of the demonstrations as well as the near-shared embodiment.

Original: Furthermore,unlike our approach, the existing (almost exclusively non-dexterous) methods require paired datafor the robot and person’s visual state spaces [33, 15, 3], assume the demonstrator and robot share avisual environment (e.g., a lab tabletop) [29, 32, 30, 31, 3], and/or tailor the imitation for a speciﬁcobject [3].
Revised: Furthermore,unlike our approach, the existing (almost exclusively non-dexterous) methods require paired datafor the robot and person’s visual state spaces [33, 15, 3], assume the demonstrator and robot share avisual environment (e.g., a lab tabletop) [29, 32, 30, 31, 3], and/or tailor the imitation for a speciﬁcobject [3].

Original: Visual object affordances As a dual to hand pose, priors for the object regions that afford aninteraction can also inﬂuence grasping.
Revised: Visual object affordances As a dual to hand pose, priors for the object regions that afford aninteraction can also inﬂuence grasping.

Original: Vision methods explore learning affordances from visualdata [36, 37, 38], though they stop short of agent action.
Revised: Vision methods explore learning affordances from visualdata [36, 37, 38], though they stop short of agent action.

Original: Visual affordances can successfully inﬂuencea pick and place robot [39, 40] and help grasping with simple grippers [9, 21, 23, 24] or a dexteroushand [8, 18].
Revised: Visual affordances can successfully inﬂuencea pick and place robot [39, 40] and help grasping with simple grippers [9, 21, 23, 24] or a dexteroushand [8, 18].

Original: Object-centric affordances predicted from images also beneﬁt the proposed model, butthey are complementary to our novel contribution—the hand-pose prior learned from video.
Revised: Object-centric affordances predicted from images also beneﬁt the proposed model, butthey are complementary to our novel contribution—the hand-pose prior learned from video.

Original: Estimating hand poses Detecting human hands and their poses is explored using a variety of visuallearning methods [41, 42, 43, 44, 45].
Revised: Estimating hand poses Detecting human hands and their poses is explored using a variety of visuallearning methods [41, 42, 43, 44, 45].

Original: Many methods jointly reason about the shape of the objectbeing grasped [43, 9, 46, 47, 46].
Revised: Many methods jointly reason about the shape of the objectbeing grasped [43, 9, 46, 47, 46].

Original: Recent work provides large-scale datasets to better understandhuman hands [48, 49, 50].
Revised: Recent work provides large-scale datasets to better understandhuman hands [48, 49, 50].

Original: We rely on the state-of-the-art FrankMocap method [44] to extract 3Dhand poses from video.
Revised: We rely on the state-of-the-art FrankMocap method [44] to extract 3Dhand poses from video.

Original: Our contribution is not a computer vision method to parse pose, but rather amachine learning framework to produce dexterous robot grasping behavior.
Revised: Our contribution is not a computer vision method to parse pose, but rather amachine learning framework to produce dexterous robot grasping behavior.

Original:  Approach
Revised:  Approach

Original: We consider the task of dexterous grasping with an articulated 30-DoF multi-ﬁngered robotic hand.
Revised: We consider the task of dexterous grasping with an articulated 30-DoF multi-ﬁngered robotic hand.

Original: Our goal is to leverage abundant human interaction videos to provide the robot with a prior overmeaningful grasp poses.
Revised: Our goal is to leverage abundant human interaction videos to provide the robot with a prior overmeaningful grasp poses.

Original: To this end, we propose D EX VIP, an approach to learn Dex terous graspingusing V ideo I nformed P ose priors.
Revised: To this end, we propose D EX VIP, an approach to learn Dex terous graspingusing V ideo I nformed P ose priors.

Original: We ﬁrst lay out the formulation of the reinforcement learningproblem for dexterous grasping (Section 3.1).
Revised: We ﬁrst lay out the formulation of the reinforcement learningproblem for dexterous grasping (Section 3.1).

Original: Then, we describe how we leverage human hand posepriors derived from in-the-wild YouTube videos for this task (Section 3.2).
Revised: Then, we describe how we leverage human hand posepriors derived from in-the-wild YouTube videos for this task (Section 3.2).

Original: 3.1 Reinforcement Learning Framework for Dexterous Grasping
Revised: 3.1 Reinforcement Learning Framework for Dexterous Grasping

Original: Background Our dexterous grasping task is structured as a reinforcement learning (RL) problem,where an agent interacts with the environment according to a policy in order to maximize a speciﬁedreward (Fig. 2).
Revised: Background Our dexterous grasping task is structured as a reinforcement learning (RL) problem,where an agent interacts with the environment according to a policy in order to maximize a speciﬁedreward (Fig. 2).

Original: At each time step t , the agent observes the current observation o t and samples anaction a t from its policy π .
Revised: At each time step t , the agent observes the current observation o t and samples anaction a t from its policy π .

Original: It then receives a scalar reward R t +1 and next observation
Revised: It then receives a scalar reward R t +1 and next observation

Original: o t +1 from theenvironment.
Revised: o t +1 from theenvironment.

Original: This feedback loop continues until the episode terminates at T time steps.
Revised: This feedback loop continues until the episode terminates at T time steps.

Original: The goal ofthe agent is to determine the optimal stochastic policy that maximizes the expected sum of rewards.
Revised: The goal ofthe agent is to determine the optimal stochastic policy that maximizes the expected sum of rewards.

Original: Observations Our task setup consists of a robotic hand positioned above a tabletop, with an objectof interest resting on the table.
Revised: Observations Our task setup consists of a robotic hand positioned above a tabletop, with an objectof interest resting on the table.

Original: At the start of each episode, we sample an object and randomlyrotate it from its canonical orientation.
Revised: At the start of each episode, we sample an object and randomlyrotate it from its canonical orientation.

Original: The observations o rt (Fig. 2, green block) at each time step tare a combination of visual and motor inputs to the robot.
Revised: The observations o rt (Fig. 2, green block) at each time step tare a combination of visual and motor inputs to the robot.

Original: The visual stream is from an egocentrichand-mounted camera.
Revised: The visual stream is from an egocentrichand-mounted camera.

Original: It consists of an RGB image of the scene I rt and the corresponding depth map
Revised: It consists of an RGB image of the scene I rt and the corresponding depth map

Original: D rt .
Revised: D rt .

Original: Additionally, we provide a binary affordance map A rt that is inferred from I r 0 using an affordance
Revised: Additionally, we provide a binary affordance map A rt that is inferred from I r 0 using an affordance

Original: I h : Human Grasp Images {Y r : Transformed
Revised: I h : Human Grasp Images {Y r : Transformed

Original: Robot Pose}
Revised: Robot Pose}

Original: R t +
Revised: R t +

Original: P h :
Revised: P h :

Original: Estimated Hand Pose
Revised: Estimated Hand Pose

Original: RGB
Revised: RGB

Original: Depth
Revised: Depth

Original: Affordance
Revised: Affordance

Original: Actor
Revised: Actor

Original: Critic
Revised: Critic

Original: Proprioception
Revised: Proprioception

Original: Hand-Obj Distance
Revised: Hand-Obj Distance

Original: Observations
Revised: Observations

Original: Policy
Revised: Policy

Original: Human Grasp Image
Revised: Human Grasp Image

Original: Estimated Human Pose
Revised: Estimated Human Pose

Original: Target Robot Pose
Revised: Target Robot Pose

Original: Pose Prior
Revised: Pose Prior

Original: I t r
Revised: I t r

Original: D t r
Revised: D t r

Original: A t r
Revised: A t r

Original: P t r d t r
Revised: P t r d t r

Original: I c*h p c*h p c*r
Revised: I c*h p c*h p c*r

Original: Environment a t r o t r
Revised: Environment a t r o t r

Original: Environment p t + 1r f V f M
Revised: Environment p t + 1r f V f M

Original: Reward
Revised: Reward

Original: Robot
Revised: Robot

Original: Pose
Revised: Pose

Original: V t
Revised: V t

Original: M t prediction network [18] to guide the agent towards functional grasp regions on the object.
Revised: M t prediction network [18] to guide the agent towards functional grasp regions on the object.

Original: The motorinputs are a combination of the robot proprioception P rt and the hand-object contact distances d rt .
Revised: The motorinputs are a combination of the robot proprioception P rt and the hand-object contact distances d rt .

Original: P rt comprises of the robot joint angles or pose p rt and angular velocities v rt of the actuator, while d rtis the pairwise distance between the object affordance regions and contact points on the hand.
Revised: P rt comprises of the robot joint angles or pose p rt and angular velocities v rt of the actuator, while d rtis the pairwise distance between the object affordance regions and contact points on the hand.

Original: Thelatter assumes the object is tracked in 3D once its affordance region is detected, following [18].
Revised: Thelatter assumes the object is tracked in 3D once its affordance region is detected, following [18].

Original: Theagent also has 21 touch sensors T r spread uniformly across the palm and ﬁngers.
Revised: Theagent also has 21 touch sensors T r spread uniformly across the palm and ﬁngers.

Original: Action space At each time step t , the policy π processes observations o rt and estimates actionsa rt —30 continuous joint angle values—which are applied at the joint angles in the actuator.
Revised: Action space At each time step t , the policy π processes observations o rt and estimates actionsa rt —30 continuous joint angle values—which are applied at the joint angles in the actuator.

Original: Therobotic manipulator we consider is the Adroit hand [51], a 30-DoF position-controlled dexterous hand.
Revised: Therobotic manipulator we consider is the Adroit hand [51], a 30-DoF position-controlled dexterous hand.

Original: With a ﬁve-ﬁngered 24-DoF actuator attached to a 6-DoF arm, the morphology of the robot handclosely resembles that of the human hand.
Revised: With a ﬁve-ﬁngered 24-DoF actuator attached to a 6-DoF arm, the morphology of the robot handclosely resembles that of the human hand.

Original: This congruence opens up an exciting avenue to infuse agrasp pose prior learned from human-object interaction videos, as we present later in Sec. 3.2.
Revised: This congruence opens up an exciting avenue to infuse agrasp pose prior learned from human-object interaction videos, as we present later in Sec. 3.2.

Original: Feature and policy learning We adopt an actor-critic model for learning the grasping policy.
Revised: Feature and policy learning We adopt an actor-critic model for learning the grasping policy.

Original: Thevisuo-motor observations o rt are processed separately using two neural networks, f V and f M (Fig.
Revised: Thevisuo-motor observations o rt are processed separately using two neural networks, f V and f M (Fig.

Original: 2,blue block).
Revised: 2,blue block).

Original: Speciﬁcally, the visual inputs encompassing { I rT , D rT , A rt } are concatenated and fedto a three-layer CNN f V that encodes them to obtain a visual embedding V t .
Revised: Speciﬁcally, the visual inputs encompassing { I rT , D rT , A rt } are concatenated and fedto a three-layer CNN f V that encodes them to obtain a visual embedding V t .

Original: The motor streamcomprised of { P rt , d rt } is processed by a two-layer fully connected network f M that encodes them toa motor embedding M t .
Revised: The motor streamcomprised of { P rt , d rt } is processed by a two-layer fully connected network f M that encodes them toa motor embedding M t .

Original: Finally V t and M t are concatenated and fed to the actor and critic networksto estimate the policy distribution π θ ( a rt | o rt ) and state values V θ
Revised: Finally V t and M t are concatenated and fed to the actor and critic networksto estimate the policy distribution π θ ( a rt | o rt ) and state values V θ

Original: ( o rt ) , respectively, at each time step.
Revised: ( o rt ) , respectively, at each time step.

Original: The resulting policy π outputs a 30-D unit-variance Gaussian whose mean is inferred by the network;we sample from this distribution to obtain the robot’s next action a rt .
Revised: The resulting policy π outputs a 30-D unit-variance Gaussian whose mean is inferred by the network;we sample from this distribution to obtain the robot’s next action a rt .

Original: We train the complete RL network with PPO [52] using a reward that encourages successful grasping,touching object affordance regions, and mimicking human hand poses, as we will detail below.
Revised: We train the complete RL network with PPO [52] using a reward that encourages successful grasping,touching object affordance regions, and mimicking human hand poses, as we will detail below.

Original: Robot hand simulator We conduct experiments in MuJoCo [53], a physics simulator commonlyused in robotics research.
Revised: Robot hand simulator We conduct experiments in MuJoCo [53], a physics simulator commonlyused in robotics research.

Original: Due to lack of access to a real robotic hand, we perform all experimentsin simulation.
Revised: Due to lack of access to a real robotic hand, we perform all experimentsin simulation.

Original: The successful transfer of dexterous policies trained purely in simulation to the realworld [11, 7, 6] supports the value of simulator-based learning in research today.
Revised: The successful transfer of dexterous policies trained purely in simulation to the realworld [11, 7, 6] supports the value of simulator-based learning in research today.

Original: In addition, weconduct numerous experiments with noisy sensing and actuation settings that might occur in the realworld to illustrate the robustness of the policy to non-ideal scenarios (see Sec. 4 and Supp.).
Revised: In addition, weconduct numerous experiments with noisy sensing and actuation settings that might occur in the realworld to illustrate the robustness of the policy to non-ideal scenarios (see Sec. 4 and Supp.).

Original: grasp poses and actions
Revised: grasp poses and actions

Original: I c*h p c*h p
Revised: I c*h p c*h p

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: a Tr I ch p ch
Revised: a Tr I ch p ch

Original: I ch
Revised: I ch

Original: [Figure]
Revised: [Figure]

Original: 3.2 Object-Speciﬁc Human Hand Pose Priors from Video
Revised: 3.2 Object-Speciﬁc Human Hand Pose Priors from Video

Original: We now describe how we leverage human-object interaction videos to enable robust dexterousgrasping.
Revised: We now describe how we leverage human-object interaction videos to enable robust dexterousgrasping.

Original: The morphological similarity between the human hand and the dexterous robot holdsimmense potential to learn meaningful pose priors for grasping.
Revised: The morphological similarity between the human hand and the dexterous robot holdsimmense potential to learn meaningful pose priors for grasping.

Original: To facilitate this, we ﬁrst curate ahuman-object interaction dataset of video frames to infer 3D grasp hand poses for a variety of objects.
Revised: To facilitate this, we ﬁrst curate ahuman-object interaction dataset of video frames to infer 3D grasp hand poses for a variety of objects.

Original: We then transfer these poses to the robotic hand using inverse kinematics, and ﬁnally develop a novel
Revised: We then transfer these poses to the robotic hand using inverse kinematics, and ﬁnally develop a novel

Original: RL reward that favors human-used poses when learning to grasp.
Revised: RL reward that favors human-used poses when learning to grasp.

Original: The proposed approach allows usto leverage readily available Internet data to learn robotic grasping.
Revised: The proposed approach allows usto leverage readily available Internet data to learn robotic grasping.

Original: Object dataset We consider household objects often encountered in daily life activity (and henceonline instructional videos) and for which 3D models are available.
Revised: Object dataset We consider household objects often encountered in daily life activity (and henceonline instructional videos) and for which 3D models are available.

Original: We acquire 3D object modelsfrom multiple public repositories: ContactDB
Revised: We acquire 3D object modelsfrom multiple public repositories: ContactDB

Original: [37], 3DNet [54], YCB [55], Free 3D [56], and 3D
Revised: [37], 3DNet [54], YCB [55], Free 3D [56], and 3D

Original: Warehouse [57].
Revised: Warehouse [57].

Original: We speciﬁcally include the 16 ContactDB objects with one-hand grasps used inrecent grasping work to facilitate concrete comparisons [18].
Revised: We speciﬁcally include the 16 ContactDB objects with one-hand grasps used inrecent grasping work to facilitate concrete comparisons [18].

Original: We obtain a total of 27 objects to beused for training the robotic grasping policy, all 16 from ContactDB plus 11 additional objects.
Revised: We obtain a total of 27 objects to beused for training the robotic grasping policy, all 16 from ContactDB plus 11 additional objects.

Original: Video frame dataset We use the HowTo100M dataset [58] to curate images containing humanobject grasps for the objects of interest.
Revised: Video frame dataset We use the HowTo100M dataset [58] to curate images containing humanobject grasps for the objects of interest.

Original: HowTo100M is a large-scale dataset consisting of 13.6 Minstructional YouTube videos across categories such as cooking, entertainment, hobbies and crafts,etc.
Revised: HowTo100M is a large-scale dataset consisting of 13.6 Minstructional YouTube videos across categories such as cooking, entertainment, hobbies and crafts,etc.

Original: We focus on videos consisting of commonly used household objects—tools and kitchen utensilssuch as mug, hammer, jug, etc.
Revised: We focus on videos consisting of commonly used household objects—tools and kitchen utensilssuch as mug, hammer, jug, etc.

Original: The idea is to capture objects in active use during natural humaninteractions so that we can obtain functional hand poses.
Revised: The idea is to capture objects in active use during natural humaninteractions so that we can obtain functional hand poses.

Original: The grasp images contain the object inits canonical upright position (e.g., pan on stove), which is also the initial vertical orientation ofthe object on the tabletop in the simulator.
Revised: The grasp images contain the object inits canonical upright position (e.g., pan on stove), which is also the initial vertical orientation ofthe object on the tabletop in the simulator.

Original: Using the above criteria, we curate an object interactionrepository
Revised: Using the above criteria, we curate an object interactionrepository

Original: I h of 715 video frames from HowTo100M where the human hand is grasping one of the 27total objects, to yield on average 26 grasp images per object.
Revised: I h of 715 video frames from HowTo100M where the human hand is grasping one of the 27total objects, to yield on average 26 grasp images per object.

Original: For instance, to collect expert data forgrasping a pan , we curate grasp images from task ids such as “ care for nonstick pans ”, “ buy cast ironpans ”, etc.
Revised: For instance, to collect expert data forgrasping a pan , we curate grasp images from task ids such as “ care for nonstick pans ”, “ buy cast ironpans ”, etc.

Original: While we found it effective to use simple ﬁlters based on the weakly labeled categoriesand speciﬁc task ids in HowTo100M, the curation step could be streamlined further by deployingvision methods for detecting hands, actions, and objects in video [48, 59, 58].
Revised: While we found it effective to use simple ﬁlters based on the weakly labeled categoriesand speciﬁc task ids in HowTo100M, the curation step could be streamlined further by deployingvision methods for detecting hands, actions, and objects in video [48, 59, 58].

Original: Target hand pose acquisition We propose to use the obtained HowTo100M grasp images I h toprovide a learning signal for robotic grasping.
Revised: Target hand pose acquisition We propose to use the obtained HowTo100M grasp images I h toprovide a learning signal for robotic grasping.

Original: To that end, for each image we ﬁrst infer its 3D humanhand pose p h .
Revised: To that end, for each image we ﬁrst infer its 3D humanhand pose p h .

Original: In particular, we employ FrankMocap [44] to estimate 3D human hand poses (see
Revised: In particular, we employ FrankMocap [44] to estimate 3D human hand poses (see

Original: Fig. 3, left).
Revised: Fig. 3, left).

Original: FrankMocap is a near real-time method for 3D hand and body pose estimation frommonocular video; it returns 3D joint angles for detected hands in each frame.
Revised: FrankMocap is a near real-time method for 3D hand and body pose estimation frommonocular video; it returns 3D joint angles for detected hands in each frame.

Original: While alternative poseestimation methods could be plugged in here, we use FrankMocap in our implementation due to itsefﬁciency and good empirical performance.
Revised: While alternative poseestimation methods could be plugged in here, we use FrankMocap in our implementation due to itsefﬁciency and good empirical performance.

Original: We keep right hand detections only, since our robot isone-handed; we leave handling bimanual grasps for future work.
Revised: We keep right hand detections only, since our robot isone-handed; we leave handling bimanual grasps for future work.

Original: This step yields a collection of different hand poses per object for a variety of objects found in thevideos.
Revised: This step yields a collection of different hand poses per object for a variety of objects found in thevideos.

Original: Let P ( c )
Revised: Let P ( c )

Original: =
Revised: =

Original: { p hc 1 , . . .
Revised: { p hc 1 , . . .

Original: , p hc n } be the set of human hand poses associated with object class c .
Revised: , p hc n } be the set of human hand poses associated with object class c .

Original: The poses within an object class are often quite consistent since the videos naturally portray people using the object in its standard functional manner, e.g., gripping a pot handle elicits the same posefor most people.
Revised: The poses within an object class are often quite consistent since the videos naturally portray people using the object in its standard functional manner, e.g., gripping a pot handle elicits the same posefor most people.

Original: However, some objects elicit a multi-modal distribution of poses (e.g., a knife heldwith or without an index ﬁnger outstretched).
Revised: However, some objects elicit a multi-modal distribution of poses (e.g., a knife heldwith or without an index ﬁnger outstretched).

Original: See Fig.
Revised: See Fig.

Original: In order to automatically discover the“consensus" pose for an object, we next apply k-medoid clustering on each set P ( c ) .
Revised: In order to automatically discover the“consensus" pose for an object, we next apply k-medoid clustering on each set P ( c ) .

Original: We consider themedoid hand pose of the largest cluster to be the consensus target hand pose p hc ∗ and use its associatedtarget robot pose p rc ∗ , obtained using a joint re-targeting mechanism described in Supp Sec D, duringpolicy learning for object c .
Revised: We consider themedoid hand pose of the largest cluster to be the consensus target hand pose p hc ∗ and use its associatedtarget robot pose p rc ∗ , obtained using a joint re-targeting mechanism described in Supp Sec D, duringpolicy learning for object c .

Original: Video-informed reward function To exert the video prior’s inﬂuence in our RL formulation, weincorporate an auxiliary reward function favoring robot poses similar to the human ones in video.
Revised: Video-informed reward function To exert the video prior’s inﬂuence in our RL formulation, weincorporate an auxiliary reward function favoring robot poses similar to the human ones in video.

Original: Inthis way, the reward function not only signals where to grasp a particular object, but also guides theagent on how to grasp effectively.
Revised: Inthis way, the reward function not only signals where to grasp a particular object, but also guides theagent on how to grasp effectively.

Original: To realize this, we combine three rewards: R succ (positive rewardwhen the object is lifted off the table), R aff (negative reward denoting the hand-affordance contactdistance obtained from [18], see Supp Sec H), and—most notably— R pose , a positive reward whenthe agent’s pose p rt matches the target grasp pose p rc ∗ for that object.
Revised: To realize this, we combine three rewards: R succ (positive rewardwhen the object is lifted off the table), R aff (negative reward denoting the hand-affordance contactdistance obtained from [18], see Supp Sec H), and—most notably— R pose , a positive reward whenthe agent’s pose p rt matches the target grasp pose p rc ∗ for that object.

Original: Our total reward function is:
Revised: Our total reward function is:

Original: [Equation]
Revised: [Equation]

Original: where α, β, γ, η are scalars weighting the rewards that are set by validation, and R entropy rewardsentropy over the target action distribution to encourage the agent to explore the action space.
Revised: where α, β, γ, η are scalars weighting the rewards that are set by validation, and R entropy rewardsentropy over the target action distribution to encourage the agent to explore the action space.

Original: Through
Revised: Through

Original: R aff , the agent is incentivized to explore areas of the object within the affordance region, while
Revised: R aff , the agent is incentivized to explore areas of the object within the affordance region, while

Original: R pose encourages the agent to reach hand states that are most suitable for grasping those regions.
Revised: R pose encourages the agent to reach hand states that are most suitable for grasping those regions.

Original: For R pose , we compute the mean per-joint angle error at time t between the robot joints p rt and thetarget grasp pose p rc ∗ .
Revised: For R pose , we compute the mean per-joint angle error at time t between the robot joints p rt and thetarget grasp pose p rc ∗ .

Original: We ignore the azimuth and elevation values of the arm in the pose error sincethey are speciﬁc to the object orientation in I h , which may be different from the robot’s viewpoint I r .
Revised: We ignore the azimuth and elevation values of the arm in the pose error sincethey are speciﬁc to the object orientation in I h , which may be different from the robot’s viewpoint I r .

Original: This provides more ﬂexibility during object reaching to orient the arm while trying to match the handpose alone.
Revised: This provides more ﬂexibility during object reaching to orient the arm while trying to match the handpose alone.

Original: Furthermore, R pose is applied only when 30% of the robot’s touch sensors are activated.
Revised: Furthermore, R pose is applied only when 30% of the robot’s touch sensors are activated.

Original: This encourages the robot to assume the target hand pose once it is close to the object and in contactwith it.
Revised: This encourages the robot to assume the target hand pose once it is close to the object and in contactwith it.

Original: Following [18], R aff is computed as the Chamfer distance between points on the hand andpoints on the inferred object affordance region.
Revised: Following [18], R aff is computed as the Chamfer distance between points on the hand andpoints on the inferred object affordance region.

Original: See Supp.
Revised: See Supp.

Original: for reward function details.
Revised: for reward function details.

Original: The proposed approach permits imitation by visual observation of the human how-to videos, yetwithout requiring access to the state-action trajectories of the human activity.
Revised: The proposed approach permits imitation by visual observation of the human how-to videos, yetwithout requiring access to the state-action trajectories of the human activity.

Original: Its mode of supervisionis therefore much lighter than that of conventional teleoperation or kinesthetic teaching, as we willsee in results.
Revised: Its mode of supervisionis therefore much lighter than that of conventional teleoperation or kinesthetic teaching, as we willsee in results.

Original: Furthermore, because our model can incorporate priors for new objects by obtainingnew training images, it scales well to add novel objects.
Revised: Furthermore, because our model can incorporate priors for new objects by obtainingnew training images, it scales well to add novel objects.

Original:  Experiments
Revised:  Experiments

Original: We present experiments to validate the impact of learning pose priors from video and to gauge
Revised: We present experiments to validate the impact of learning pose priors from video and to gauge

Original: D EX VIP’s performance relative to existing methods and baselines.
Revised: D EX VIP’s performance relative to existing methods and baselines.

Original: Compared methods We compare to the following methods: ( 1 ) C O M: uses the center of mass ofthe object as a grasp location prior, since this location can lead to stable grasps [60].
Revised: Compared methods We compare to the following methods: ( 1 ) C O M: uses the center of mass ofthe object as a grasp location prior, since this location can lead to stable grasps [60].

Original: We implementthis by penalizing the hand-CoM distance for R aff in Eqn 1, and removing R pose .
Revised: We implementthis by penalizing the hand-CoM distance for R aff in Eqn 1, and removing R pose .

Original: ( 2 ) T OUCH :uses only the touch sensors T r on the hand to positively reward the agent +1 for object interaction,when 30% of them are activated, but imposes no supervision on the hand pose.
Revised: ( 2 ) T OUCH :uses only the touch sensors T r on the hand to positively reward the agent +1 for object interaction,when 30% of them are activated, but imposes no supervision on the hand pose.

Original: ( 3 ) GRAFF [18]:a state-of-the-art RL grasping model that trains a policy to grasp objects at inferred object-centricaffordance regions.
Revised: ( 3 ) GRAFF [18]:a state-of-the-art RL grasping model that trains a policy to grasp objects at inferred object-centricaffordance regions.

Original: Unlike our method, GRAFF does not enforce any prior over the agent’s handpose.
Revised: Unlike our method, GRAFF does not enforce any prior over the agent’s handpose.

Original: All the above three RL methods use the same architecture as our model for the graspingpolicy, allowing for a fair comparison.
Revised: All the above three RL methods use the same architecture as our model for the graspingpolicy, allowing for a fair comparison.

Original: ( 4 ) DAPG [2]: is a hybrid imitation+RL model that usesmotion-glove demonstrations collected from a human expert in VR.
Revised: ( 4 ) DAPG [2]: is a hybrid imitation+RL model that usesmotion-glove demonstrations collected from a human expert in VR.

Original: It is trained with object-speciﬁcmocap demonstrations collected by [18] for grasping ContactDB object (25 demos per object).
Revised: It is trained with object-speciﬁcmocap demonstrations collected by [18] for grasping ContactDB object (25 demos per object).

Original: Forobjects beyond ones in ContactDB, we use demos from the object most similar in shape in ContactDB.
Revised: Forobjects beyond ones in ContactDB, we use demos from the object most similar in shape in ContactDB.

Original: Metrics We report four metrics: ( 1) Grasp Success: when the object is lifted off the table by the handfor at least the last 50 time steps (a quarter of the episode length) to allow time to reach the object andpick it up.
Revised: Metrics We report four metrics: ( 1) Grasp Success: when the object is lifted off the table by the handfor at least the last 50 time steps (a quarter of the episode length) to allow time to reach the object andpick it up.

Original: ( 2) Grasp Stability: the ﬁrmness with which the object is held by the robot, discountinggrasps in which the object can easily be dropped.
Revised: ( 2) Grasp Stability: the ﬁrmness with which the object is held by the robot, discountinggrasps in which the object can easily be dropped.

Original: We apply perturbation forces of 1 Newton in sixorthogonal directions on the object after an episode completes.
Revised: We apply perturbation forces of 1 Newton in sixorthogonal directions on the object after an episode completes.

Original: If the object continues to be grasped by the agent, the grasp is deemed stable
Revised: If the object continues to be grasped by the agent, the grasp is deemed stable

Original: Functionality percentage of successful grasps
Revised: Functionality percentage of successful grasps

Original: which the hand lies close to the GT affordance region.
Revised: which the hand lies close to the GT affordance region.

Original: This metric evaluates the utility of the grasp post-
Revised: This metric evaluates the utility of the grasp post-

Original: : the hand pose after a successful grasp p rT .
Revised: : the hand pose after a successful grasp p rT .

Original: It tells us how human-like the learned are.
Revised: It tells us how human-like the learned are.

Original: We normalize all metrics on a [0 , 100%] scale, where higher is better.
Revised: We normalize all metrics on a [0 , 100%] scale, where higher is better.

Original: We evaluate 100 episodes perobject with the objects placed at different initial orientations ranging from [0,180 ◦ ].
Revised: We evaluate 100 episodes perobject with the objects placed at different initial orientations ranging from [0,180 ◦ ].

Original: We report themean and standard deviation of the metrics across all models trained with four random seeds.
Revised: We report themean and standard deviation of the metrics across all models trained with four random seeds.

Original: Implementation details The visual encoder f V has ﬁlters of size [8,4,3], and a bottleneck 512-Dand uses ReLU activations.
Revised: Implementation details The visual encoder f V has ﬁlters of size [8,4,3], and a bottleneck 512-Dand uses ReLU activations.

Original: The motor encoder f M has dimensions [512,512].
Revised: The motor encoder f M has dimensions [512,512].

Original: For the hand-objectcontacts, we use 10 and 20 uniformly sampled points on the hand and affordance region, respectively,following [18].
Revised: For the hand-objectcontacts, we use 10 and 20 uniformly sampled points on the hand and affordance region, respectively,following [18].

Original: The entire network is optimized using Adam with a learning rate of 5 e − 5 .
Revised: The entire network is optimized using Adam with a learning rate of 5 e − 5 .

Original: A singlegrasping policy is trained on all the curated objects for 150M agent steps with an episode length of 200time steps.
Revised: A singlegrasping policy is trained on all the curated objects for 150M agent steps with an episode length of 200time steps.

Original: The coefﬁcients in the reward function (Eq. 1) are set as: α = 1 , β = 1 , γ = 1 , η = 0 .
Revised: The coefﬁcients in the reward function (Eq. 1) are set as: α = 1 , β = 1 , γ = 1 , η = 0 .

Original: We train for four random seed initializations.
Revised: We train for four random seed initializations.

Original: All project code will be made publicly available.
Revised: All project code will be made publicly available.

Original: Grasping policy performance We take the policy trained on all 27 objects and ﬁrst evaluate it onthe 16 objects from ContactDB
Revised: Grasping policy performance We take the policy trained on all 27 objects and ﬁrst evaluate it onthe 16 objects from ContactDB

Original: Since these objects have ground truth affordances (used whentraining GRAFF and our model) and mocap demonstrations (used when training DAPG), theyrepresent the best case scenario for the existing models to train with clean expert data.
Revised: Since these objects have ground truth affordances (used whentraining GRAFF and our model) and mocap demonstrations (used when training DAPG), theyrepresent the best case scenario for the existing models to train with clean expert data.

Original: Note that ourmethod always uses hand poses inferred from YouTube videos, even for the ContactDB objects.
Revised: Note that ourmethod always uses hand poses inferred from YouTube videos, even for the ContactDB objects.

Original: N o P r i o r
Revised: N o P r i o r

Original: C o M
Revised: C o M

Original: G R A
Revised: G R A

Original: FF
Revised: FF

Original: D A P G
Revised: D A P G

Revised: Pan:

Original: [Figure]
Revised: [Figure]

Revised: _all_seed1 Mug:

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: seed
Revised: _seed

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original:  [Figure][Figure] _all
Revised: _all_seed [Figure]

Original: [Figure]
Revised: [Figure]

Original: seed
Revised: _all_seed

Original: D e x V I P
Revised: D e x V I P

Original: T o u c h pan mug teapot knife cup hammer knife
Revised: T o u c h pan mug teapot knife cup hammer knife

Original: [Figure]
Revised: [Figure]

Original: frames for the grasping task.
Revised: frames for the grasping task.

Original: Our D EX VIP policy guided by pose-priors is able to successfully grasp objects in natural human-like poses while the other methods may either generate unusual poses or fail to grasp effectively.
Revised: Our D EX VIP policy guided by pose-priors is able to successfully grasp objects in natural human-like poses while the other methods may either generate unusual poses or fail to grasp effectively.

Original: Please see Supp. video.
Revised: Please see Supp. video.

Original: Fig 5 (left) shows the results.
Revised: Fig 5 (left) shows the results.

Original: D EX VIP consistently outperforms all the methods on all metrics.
Revised: D EX VIP consistently outperforms all the methods on all metrics.

Original: The grasp success and stability rates experience a signiﬁcant boost even compared to
Revised: The grasp success and stability rates experience a signiﬁcant boost even compared to

Original: GRAFF [18], which utilizes object affordancesbut does not enforce any constraints on the handpose.
Revised: GRAFF [18], which utilizes object affordancesbut does not enforce any constraints on the handpose.

Original: The Functionality values are similar asboth the methods encourage the agent to graspthe object at the affordance regions.
Revised: The Functionality values are similar asboth the methods encourage the agent to graspthe object at the affordance regions.

Original: Our methodalso scores well on the Posture metric, indicating that the learned policies indeed demonstratehuman-like behavior during grasping.
Revised: Our methodalso scores well on the Posture metric, indicating that the learned policies indeed demonstratehuman-like behavior during grasping.

Original: See
Revised: See

Original: Supp.
Revised: Supp.

Revised: for results on non-ContactDB objects and

Original: for TSNE plots illustrating our model’shuman-like poses.
Revised: TSNE plots illustrating our model’s human-likeposes.

Original: Fig.
Revised: Fig.

Original: 4 shows grasp policies from sampleepisodes.
Revised: 4 shows grasp policies from sampleepisodes.

Original: We can see that D EX VIP is able tograsp objects with a human-like natural posturecompared to the other methods.
Revised: We can see that D EX VIP is able tograsp objects with a human-like natural posturecompared to the other methods.

Original: Contrast thiswith C O M or GRAFF, which only uses objectaffordances: those agents often end up in unusual poses which reduce their applicability to post-grasp functional tasks using the object.
Revised: Contrast thiswith C O M or GRAFF, which only uses objectaffordances: those agents often end up in un- usual poses which reduce their applicability to post-grasp functional tasks using the object.

Original: Failure cases arise when some objects are in orientationsnot amenable to the target hand pose.
Revised: Failurecases arise when some objects are in orientations not amenable to the target hand pose.

Original: For instance, a knife with the handle on the left would ideallybe picked up differently and then re-oriented for use.
Revised: For instance,a knife with the handle on the left would ideally be picked up differently and then re-oriented for use.

Original: Training speed Fig.
Revised: Training speed Fig.

Original: 5 (right) shows the training curves.
Revised: 5 (right) shows the training curves.

Original: Our method learns successful policies 20%faster than the next best method, GRAFF [18].
Revised: Our method learns successful policies 20%faster than the next best method, GRAFF [18].

Original: This underscores how the hand pose priors enableour agent to approach objects in easily graspable conﬁgurations, thus improving sample efﬁciency.
Revised: This underscores how the hand pose priors enableour agent to approach objects in easily graspable conﬁgurations, thus improving sample efﬁciency.

Original: Expert data: teleoperation vs. YouTube video Compared to traditional demonstrations, a keyadvantage of our video-based approach is the ease of data collection and scalability to new objects.
Revised: Expert data: teleoperation vs. YouTube video Compared to traditional demonstrations, a keyadvantage of our video-based approach is the ease of data collection and scalability to new objects.

Original: We analyze the time taken to collect demonstrations for DAPG [2] versus the time taken to curatevideos for D EX VIP.
Revised: We analyze the time taken to collect demonstrations for DAPG [2] versus the time taken to curatevideos for D EX VIP.

Original: On average, it takes 5 minutes to collect a single demonstration for DAPGowing to the complex setup, while a video or image for D EX VIP is collected in a few seconds.
Revised: On average, it takes 5 minutes to collect a single demonstration for DAPGowing to the complex setup, while a video or image for D EX VIP is collected in a few seconds.

Original: Fig.
Revised: Fig.

Original: 6 quantiﬁes the impact of the efﬁciency of human demonstrations for our model (trained withvideo frames) compared to traditional state-action demonstrations (trained with VR demos).
Revised: 6 quantiﬁes the impact of the efﬁciency of human demonstrations for our model (trained withvideo frames) compared to traditional state-action demonstrations (trained with VR demos).

Original: Plottingsuccess rates as a function of accumulated demo experience, we see how quickly the proposed imagebased supervision translates to grasping success on an increasing number of new objects, whereaswith traditional demonstrations reaching peak performance takes much longer.
Revised: Plottingsuccess rates as a function of accumulated demo experience, we see how quickly the proposed imagebased supervision translates to grasping success on an increasing number of new objects, whereaswith traditional demonstrations reaching peak performance takes much longer.

Original: This highlights thesigniﬁcant gains that can be realized by shifting from tele-op to video supervision for robot learning.
Revised: This highlights thesigniﬁcant gains that can be realized by shifting from tele-op to video supervision for robot learning.

Original: DexVIP beneﬁts from easily available Internet data to scale up supervision.
Revised: DexVIP beneﬁts from easily available Internet data to scale up supervision.

Original: This efﬁciency also means new objects are quick toadd.
Revised: This efﬁciency also means new objects are quick toadd.

Original: To illustrate, we further evaluate D EX VIP and
Revised: To illustrate, we further evaluate D EX VIP and

Original: DAPG on all 11 non-ContactDB objects, for which mocap demonstrations are not available.
Revised: DAPG on all 11 non-ContactDB objects, for which mocap demonstrations are not available.

Original: DAPG’s successrate drops from 59% to 50% —a 15% relative drop inperformance—while D EX VIP experiences a marginal4% drop from 68% to 65% , remaining comparable toits performance on ContactDB.
Revised: DAPG’s successrate drops from 59% to 50% —a 15% relative drop inperformance—while D EX VIP experiences a marginal4% drop from 68% to 65% , remaining comparable toits performance on ContactDB.

Original: Since DAPG is trainedon sub-optimal demonstrations, this precludes it fromgeneralizing well to objects for which expert data is absent.
Revised: Since DAPG is trainedon sub-optimal demonstrations, this precludes it fromgeneralizing well to objects for which expert data is absent.

Original: D EX VIP, on the other hand, is able to beneﬁt fromeasily available Internet images to scale up supervision.
Revised: D EX VIP, on the other hand, is able to beneﬁt fromeasily available Internet images to scale up supervision.

Original: Ablations Next we investigate how different components of the reward inﬂuence the dexterous graspingpolicy.
Revised: Ablations Next we investigate how different components of the reward inﬂuence the dexterous graspingpolicy.

Original: Using only R aff , the success rate is 60% on
Revised: Using only R aff , the success rate is 60% on

Original: ContactDB.
Revised: ContactDB.

Original: When we successively add touch and hand pose priors to the reward function (Eq. 1), weobtain success rates of 63% and 68% respectively.
Revised: When we successively add touch and hand pose priors to the reward function (Eq. 1), weobtain success rates of 63% and 68% respectively.

Original: Thus our full model is the most effective.
Revised: Thus our full model is the most effective.

Original: Noisy sensing and actuation Finally, to mimic non-ideal real-world scenarios, we induce multiplesources of noise into our agent’s sensory and actuation modules during training and testing followingprior work [11, 7, 6, 18].
Revised: Noisy sensing and actuation Finally, to mimic non-ideal real-world scenarios, we induce multiplesources of noise into our agent’s sensory and actuation modules during training and testing followingprior work [11, 7, 6, 18].

Original: These include Gaussian noise on the robot’s proprioception P rt , objecttracking d rt , and actuation a rt , as well as pixel perturbations on the image observations I rt .
Revised: These include Gaussian noise on the robot’s proprioception P rt , objecttracking d rt , and actuation a rt , as well as pixel perturbations on the image observations I rt .

Original: Underheavy noise, D EX VIP still yields a grasp success rate of 64% , even outperforming noise-free modelsof the other methods (Fig. 5, shaded bars).
Revised: Underheavy noise, D EX VIP still yields a grasp success rate of 64% , even outperforming noise-free modelsof the other methods (Fig. 5, shaded bars).

Original: Despite the lack of access to a real robot, this encouragingresult in a noise-induced simulation environment lends support for potentially transferring the learnedpolicies to the real world [6, 7, 61] were we to gain access to a dexterous robot.
Revised: Despite the lack of access to a real robot, this encouragingresult in a noise-induced simulation environment lends support for potentially transferring the learnedpolicies to the real world [6, 7, 61] were we to gain access to a dexterous robot.

Original: Please see Supp.
Revised: Please see Supp.

Original: formore details.
Revised: formore details.

Original:  Conclusion
Revised:  Conclusion

Original: We proposed an approach to learn dexterous robotic grasping from human hand pose priors derivedfrom video.
Revised: We proposed an approach to learn dexterous robotic grasping from human hand pose priors derivedfrom video.

Original: By leveraging human-object interactions in YouTube videos, we showed that our dexterous grasping policies outperformed methods that did not have access to these priors, includingtwo state-of-the-art models for RL-based grasping.
Revised: By leveraging human-object interactions in YouTube videos, we showed that our dexterous grasping policies outperformed methods that did not have access to these priors, includingtwo state-of-the-art models for RL-based grasping.

Original: Key advantages of our approach are 1) humansare observed directly doing real object interactions, without the interference of conventional demonstration tools; and 2) expert information from video sources scales well with new objects.
Revised: Key advantages of our approach are 1) humansare observed directly doing real object interactions, without the interference of conventional demonstration tools; and 2) expert information from video sources scales well with new objects.

Original: Thisis an encouraging step towards training robotic manipulation agents from weakly supervised andeasily scalable in-the-wild expert data available on the Internet.
Revised: Thisis an encouraging step towards training robotic manipulation agents from weakly supervised andeasily scalable in-the-wild expert data available on the Internet.

Original: In the future, we are interested inexpanding the repertoire of tasks beyond grasping to learn ﬁne-grained manipulation skills fromhuman interaction videos.
Revised: In the future, we are interested inexpanding the repertoire of tasks beyond grasping to learn ﬁne-grained manipulation skills fromhuman interaction videos.