Patent ID: 8397181

Claim:
A method for marking a position of a real world object on a see-through display comprising: capturing an image of a real world object with an imaging device; identifying a pointing device within the field of view of the imaging device; determining a viewing angle along a line from a see-through display to a tip of the pointing device to the real world object based on an azimuth angle and an elevation angle between locations of the real world object and the pointing device within the field of view of the imaging device; steering a ranging device toward the real world object based on the determined viewing angle; determining a real world distance between the ranging device and the real world object with the ranging device, wherein the ranging device determines the real world distance based on an amount of time it takes a signal to travel from the ranging device to the real world object and back to the ranging device; determining an orientation of the see-through display; calculating a real world position of the real world object based on the viewing angle to the real world object, the real world distance between the ranging device and the real world object, and the orientation of the see-through display; determining a first location on the see-through display that corresponds to the calculated real world position of the real world object; displaying a mark on the see-through display at the first location that corresponds to the calculated real world position of the real world object; tracking movement of the see-through display relative to the calculated real world position of the real world object using an inertial sensor; and adjusting the first location of the mark as displayed on the see-through display to account for the movement of the see-through display relative to the calculated real world position of the real world object.