Patent ID: 8913037

Claim:
An apparatus comprising; a time-of-flight (ToF) sensor configured to receive light from an illuminated portion of an environment and determine a ToF value associated with one or more objects in the illuminated portion of the environment; an image sensor configured to capture color images of at least a portion of the environment; a projector configured to project content onto a display surface within the environment, at least a portion of the content having multiple selectable elements; a processor; and computer-readable media storing computer-executable instructions that, when executed by the processor, cause the processor to perform acts comprising: computing depth information associated with the one or more objects based at least in part on the ToF value; determining, from the depth information, when a hand or finger of a user touches or is within a threshold distance of the display surface; and upon determining that the hand or finger of the user touches or is within the threshold distance, determining which element of the multiple selectable elements the hand or finger of the user has selected, the determining including identifying changes between a first of the color images captured prior to the hand or finger touching or being within the threshold distance and a second of the color images captured after the hand or finger touches or is within the threshold distance.