PixelAttentive Policy Gradient for MultiFingered Grasping in Cluttered Scenes Recent advances in onpolicy reinforcement learning (RL) methods enabled learning agents in virtual environments to master complex tasks with highdimensional and continuous observation and action spaces. However, leveraging this family of algorithms in multifingered robotic grasping remains a challenge due to large simtoreal fidelity gaps and the high sample complexity of onpolicy RL algorithms. This work aims to bridge these gaps by first reinforcementlearning a multifingered robotic grasping policy in simulation that operates in the pixel space of the input: a single depth image. Using a mapping from pixel space to Cartesian space according to the depth map, this method transfers to the real world with high fidelity and introduces a novel attention mechanism that substantially improves grasp success rate in cluttered environments. Finally, the directgenerative nature of this method allows learning of multifingered grasps that have flexible endeffector positions, orientations and rotations, as well as all degrees of freedom of the hand.

A Comparison of Action Spaces for Learning Manipulation Tasks Designing reinforcement learning (RL) problems that can produce delicate and precise manipulation policies requires careful choice of the reward function, state, and action spaces. Much prior work on applying RL to manipulation tasks has defined the action space in terms of direct joint torques or reference positions for a jointspace proportional derivative (PD) controller. In practice, it is often possible to add additional structure by taking advantage of modelbased controllers that support both accurate positioning and control of the dynamic response of the manipulator. In this paper, we evaluate how the choice of action space for dynamic manipulation tasks affects the sample complexity as well as the final quality of learned policies. We compare learning performance across three tasks (peg insertion, hammering, and pushing), four action spaces (torque, joint PD, inverse dynamics, and impedance control), and using two modern reinforcement learning algorithms (Proximal Policy optimization and Soft ActorCritic). Our results lend support to the hypothesis that learning references for a taskspace impedance controller significantly reduces the number of samples needed to achieve good performance across all tasks and algorithms.