Patent Document ID: 9304582
Application ID: 14134721
Patent Status: 1

Claim One:
1. A system, comprising: a depth sensor configured to produce a depth image of a scene, the scene containing one or more surfaces, wherein the depth image indicates three-dimensional coordinates for the one or more surfaces; a color camera configured to produce a camera image of the scene, the camera image comprising a camera color value for an individual one of two-dimensional camera coordinates; a projector configured to project a display image onto the scene, the display image comprising a projector color value for an individual one of two-dimensional projector coordinates; and operating logic configured to perform acts comprising: obtaining coordinate transformations between the three-dimensional coordinates, the two-dimensional camera coordinates, and the two-dimensional projector coordinates; analyzing the depth image to detect an item within the scene and to identify a set of the surface points that are formed by the detected item; identifying a set of camera color values that correspond to the item based at least in part on the three-dimensional coordinates of surface points of the set of surface points and one or more of the coordinate transformations; analyzing the set of camera color values to determine a color of the item; and identifying a set of the projector color values that correspond to the item based at least in part on the three-dimensional coordinates of the surface points of the set of surface points and one or more of the coordinate transformations.