title: Enhancing Robot Manipulation Skills through Learning
Creators: name: Müller Carvalho, João André
Publisher: name: UNSPECIFIED
Year: 2025

description: Robotic manipulation is a core area in robotics that focuses on developing systems that enable robots to interact and modify their environment by manipulating objects in the physical world. From stacking groceries on supermarket shelves to solving difficult assembly tasks in industrial settings, robots are expected to handle multiple manipulation challenges in different environments and handle various objects. Manipulation systems are composed of several modules, such as perception, motion planning, and control, each specialized in solving a part of the task, enabling researchers to focus on specific problems and advance the field in a modular way. However, these modules are still nowadays heavily-engineered to solve a specific problem in a static environment or handle particular pre-determined object shapes and sizes. If we expect future robots to be more present in our factories and our homes, they must be able to adapt to new environments and objects and learn new tasks without being constantly reprogrammed by expert users. Noticing the limitations of current manipulation systems and the great advancements in machine learning over the last decade, it is natural to ask how learning methods can help create more capable and adaptive robots. In this thesis, we ask how learning methods can be used to improve the performance of the robotic manipulation pipeline in the subproblems of motion planning, grasping, and insertion.

In three separate chapters, we explore different learning approaches to these subproblems.

First, we note that the performance of optimization-based robot motion planning algorithms highly depends on initial samples, commonly obtained by running a sample-based planning method, which can be slow in high-dimensional and complex scenes and produce non-smooth solutions. Therefore, given previously solved path-planning problems, it is highly desirable to learn this data distribution and use it as a prior for new similar problems. However, this data is highly multimodal, as there are many ways to traverse an environment from a start to a goal configuration. To encode this prior knowledge and adapt to new tasks, we propose Motion Planning Diffusion (MPD), a method to learn trajectory priors for motion planning with diffusion models that, given a new cost function, adapts to new scenes and generates diverse and smooth solutions.

Second, we explore the problem of grasping objects given a single-view camera. The problem of stable grasping is encountered in almost every manipulation task, as the first step in manipulating an object is to grasp it. Particularly in real-world applications, there is often only partial information about the object due to occlusions or sensor limitations. As obtaining real-world grasp data is costly and time-consuming, simulation methods have been developed to collect large amounts of data pairs of objects and stable grasps. The generated data is highly multimodal, as there are many ways to grasp an object. To overcome these challenges, we propose Grasp Diffusion Network (GDN), a method that learns a multimodal grasp generative model given partial views of an object, by modelling the distribution with diffusion models in the manifold of homogenous transformations.

Third, we combine imitation and reinforcement learning to learn control policies for assembly tasks in environments that are difficult to model, such as contact-rich insertion tasks. To solve these problems in the real world, it is helpful to first provide a small set of demonstrations and learn a trajectory distribution with imitation learning. However, sampling a trajectory from the learned distribution and replaying the movement might not solve the task for several reasons. On the one hand, the sampled trajectory might be slightly off. On the other hand, in contact-rich tasks, a low-gain controller is commonly used, and it might not follow the trajectory precisely. Therefore, given a handful of trajectory demonstrations from a human performing an insertion task, we propose to learn Probabilistic Movement Primitives and later adapt sampled trajectories with Residual Reinforcement Learning to account for an imprecise controller and corrections in position and orientation during task execution. The proposed method is evaluated in a real-world block insertion task using a 7-dof robot that the nominal policy previously failed to solve.

In summary, this thesis contributes to advancing the field of robotic manipulation by proposing learning methods that handle large amounts of multimodal data, model the distribution in the correct manifold, and improve sample efficiency when learning from trial-and-error in contact-rich tasks. For each approach, we provide experiments in simulation and real-world tasks that show our methods can improve the performance of different components of the manipulation stack, from motion planning, grasping, and assembly tasks, and thus are a step forward in building more capable and adaptive learning-based robots.