Patent Document ID: 9144905
Application ID: 14160417
Patent Status: 1

Claim One:
1. A system for identifying functional elements of an object for robotic manipulation, the system comprising one or more processors and a non-transitory computer-readable medium having executable instructions encoded thereon such that when the instructions are executed, the one or more processors perform operations of: causing a robot to repeatedly manipulate an object, wherein the robot comprises an audio sensor and a plurality of touch sensors; recording a three-dimensional (3D) location of each audio event that produces a response as measured by the audio sensor during manipulation of the object; recording a 3D location of each tactile event that produces a response as detected by the plurality of touch sensors during manipulation of the object; determining a 3D location of each audio and tactile event in 3D space; generating a 3D audio point cloud denoting the 3D locations of the audio events; generating a 3D tactile point cloud denoting the 3D locations of the tactile events; registering the 3D audio point cloud and the 3D tactile point cloud with a 3D model point cloud of the object; and annotating the 3D model point cloud of the object with the audio events to generate an annotated 3D model point cloud of the object, which encodes the location of a functional element of the object.