ThinkGrasp: A Vision-Language System for Strategic Part Grasping in
  Clutter
Authors: name: Yaoyao Qian; name: Xupeng Zhu; name: Ondrej Biza; name: Shuo Jiang; name: Linfeng Zhao; name: Haojie Huang; name: Yu Qi; name: Robert Platt
Published: 2024-07-16 01:06:33

Abstract: Robotic grasping in cluttered environments remains a significant challenge
due to occlusions and complex object arrangements. We have developed
ThinkGrasp, a plug-and-play vision-language grasping system that makes use of
GPT-4o's advanced contextual reasoning for heavy clutter environment grasping
strategies. ThinkGrasp can effectively identify and generate grasp poses for
target objects, even when they are heavily obstructed or nearly invisible, by
using goal-oriented language to guide the removal of obstructing objects. This
approach progressively uncovers the target object and ultimately grasps it with
a few steps and a high success rate. In both simulated and real experiments,
ThinkGrasp achieved a high success rate and significantly outperformed
state-of-the-art methods in heavily cluttered environments or with diverse
unseen objects, demonstrating strong generalization capabilities.