Planning Reactive Manipulation in Dynamic Environments When robots perform manipulation tasks, they need to determine their own movement, as well as how to make and break contact with objects in their environment. Reasoning about the motions of robots and objects simultaneously leads to a constrained planning problem in a highdimensional statespace. Additionally, when environments change dynamically motions must be computed in realtime.To this end, we propose a feedback planner for manipulation. We model manipulation as constrained motion and use this model to automatically derive a set of constraintbased controllers. These controllers are used in a switchingcontrol scheme, where the active controller is chosen by a reinforcement learning agent. Our approach is capable of addressing tasks with secondorder dynamics, closed kinematic chains, and timevariant environments. We validated our approach in simulation and on a real, dualarm robot. Extensive simulation of three distinct robots and tasks show a significant increase in robustness compared to a previous approach.

A Comparison of Action Spaces for Learning Manipulation Tasks Designing reinforcement learning (RL) problems that can produce delicate and precise manipulation policies requires careful choice of the reward function, state, and action spaces. Much prior work on applying RL to manipulation tasks has defined the action space in terms of direct joint torques or reference positions for a jointspace proportional derivative (PD) controller. In practice, it is often possible to add additional structure by taking advantage of modelbased controllers that support both accurate positioning and control of the dynamic response of the manipulator. In this paper, we evaluate how the choice of action space for dynamic manipulation tasks affects the sample complexity as well as the final quality of learned policies. We compare learning performance across three tasks (peg insertion, hammering, and pushing), four action spaces (torque, joint PD, inverse dynamics, and impedance control), and using two modern reinforcement learning algorithms (Proximal Policy optimization and Soft ActorCritic). Our results lend support to the hypothesis that learning references for a taskspace impedance controller significantly reduces the number of samples needed to achieve good performance across all tasks and algorithms.