We address the manipulation task of retrieving a target object from a cluttered shelf. When the target object is hidden, the robot must search through the clutter for retrieving it. Solving this task requires reasoning over the likely locations of the target object. It also requires physics reasoning over multi-object interactions and future occlusions. In this work, we present a data-driven approach for generating occlusion-aware actions in closed-loop. We present a hybrid planner that explores likely states generated from a learned distribution over the location of the target object. The search is guided by a heuristic trained with reinforcement learning to evaluate occluded observations . We evaluate our approach in different environments with varying clutter densities and physics parameters . The results validate that our approach can search and retrieve a target object in different physics environments, while only being trained in simulation . It achieves near real-time behaviour with a success rate exceeding 88\% .
We address the manipulation task of retrieving a target object from a cluttered shelf. When the target object is hidden, the robot must search through the clutter for retrieving it. Solving this task requires reasoning over the likely locations of the target object. It also requires physics reasoning over multi-object interactions and future occlusions. In this work, we present a data-driven approach for generating occlusion-aware actions in closed-loop. We present a hybrid planner that explores likely states generated from a learned distribution over the location of the target object. The search is guided by a heuristic trained with reinforcement learning to evaluate observations with occlusions . We evaluate our approach in different simulation and real-world settings . The results validate that our approach can search and retrieve a target object in near real time in the real world while only being trained in simulation .
R
occluded observations
observations with occlusions
R
environments with varying clutter densities and physics parameters
simulation and real-world settings
R
different physics environments,
near real time in the real world
D
. It achieves near real-time behaviour with a success rate exceeding 88\%