Patent ID: 8117544

Claim:
A method for identifying and tracking a pointing indicator in an input video stream, the method comprising: receiving an input video stream of a computer display from a client computer at a video proxy device, wherein the video proxy device sends an output video stream to a display device; extracting a plurality of frames from the received the input video stream, each frame being located at a predetermined time before a previous frame in the input video stream; detecting coordinates indicative of a position of a graphical pointing indicator in each of the plurality of frames from the input video stream during presentation of the input video stream at the video proxy device, wherein the graphical pointing indicator is within the input video stream and is controlled from the client computer; detecting movement of the graphical pointing indicator among the plurality of frames during presentation of the input video stream at the video proxy device, wherein detecting movement of the graphical pointing indicator at the video proxy device includes maintaining coordinates of the graphical pointing indicator for each of the plurality of frames, storing the coordinates of the graphical pointing indicator for each frame, comparing, for each frame, the stored coordinates of the graphical pointing indicator, wherein the position of the graphical pointing indicator for one of the plurality of frames is compared to a subsequent one of the plurality of frames, and wherein a particular detected movement of the graphical pointing indicator, controlled from the client computer, invokes control actions executed by the video proxy device, and wherein the control actions include making at least one change to the output video stream; determining when the graphical pointing indicator is moving in a particular trajectory relative to an item shown in the input video stream; determining that the graphical pointing indicator has completed the particular trajectory and identifying a control action associated with the particular trajectory; executing the control action to produce the output video stream; and sending the output video stream to the display device.