Patent Document ID: 9987744
Application ID: 15592849
Patent Status: 1

Claim One:
1. A method implemented by one or more processors, comprising: providing, for display on a client computing device, an image capturing at least a portion of an object in an environment, the image being generated based on image data generated by one or more cameras that are associated with a robot and that are viewing the environment; receiving selection data in response to providing the image, the selection data indicating one or more first selected pixels of the image and one or more second selected pixels of the image, the first selected pixels being selected based on a first selection by a user via a user interface input device of the client computing device, and the second selected pixels being selected based on a second selection by the user via the user interface input device; selecting, based on the first selected pixels and the second selected pixels, at least one particular three-dimensional (3D) point from a group of 3D points, wherein the group of 3D points is generated based on the image data generated by the one or more cameras or determined based on additional sensor data generated by a 3D scanner that is associated with the robot and that is viewing the environment; determining a grasp approach vector based on the particular 3D point; determining a grasp pose that conforms to the grasp approach vector; and providing one or more control commands that cause a grasping end effector of the robot to traverse to the grasp pose and attempt a grasp of the object subsequent to traversing to the grasp pose.