Patent Document ID: 10037312
Application ID: 14667529
Patent Status: 1

Claim One:
1. A gaze annotation method for an image, comprising: at a mobile device, wherein the mobile device is coupled with a scene camera, a display, an audio input device and a gaze tracking device that are disposed locally and in proximity to each other: displaying on the display a live image received from the scene camera of a scene in proximity to a first user; receiving a first user command from the first user to capture the live image from the scene camera; while displaying the captured live image, receiving a second user command from the first user to create an annotation for the displayed live image; in response to the second user command: identifying from information recorded by the gaze tracking device a first point-of-regard estimating the first user's gaze in the displayed live image; receiving a spoken annotation captured locally by the audio input device from the first user; transcribing the spoken annotation using automatic speech recognition; from the transcribed spoken annotation, identifying a keyword indicative of a deictic reference to the first point-of-regard; associating the first point-of-regard with the deictic reference, including aligning the keyword with the first point-of-regard of the first user's gaze in the live image; and displaying on the live image a deictic gaze marker as a visualization of the deictic reference.