Patent ID: 8860760

Claim:
An augmented reality (AR) system, comprising: a camera for recording images within a field-of-view (FOV) scanned over a larger field-of-regard (FOR) in a scene including parts; a display configured to display images of the scene in a FOV overlapping the camera FOV; one or more memory units; templates for the parts stored in said one or more memory units; a plurality of instructions stored in said one or more memory units, each said instruction including a list of parts and instructions for the use of those parts; and a computer comprising: a classifier module that processes images of the scene using the templates to locate and identify the parts in the scene over the FOR, said classified parts' identities and locations relative to the FOR stored in said one or more memory units and retained in said memory units for parts that lie both inside the current FOV and outside the current FOV stored in said one or more memory units; a pose module that processes data from images of the scene to estimate camera pose relative to the FOR; an instruction module that retrieves an instruction and a list of all parts relevant to the instruction; a tracker module that for each instruction determines from the list of all parts relevant to the instruction, locations of those parts relative to the FOR and the estimated camera pose relative to the FOR, the list of parts and the locations of those parts that lie within the current FOV and those parts that lie outside the current FOV; a parts rendering module that for each instruction receives the list of parts within the current FOV and a pose for each of those parts with respect to the camera and renders parts icons registered to parts within the current FOV and receives the list of parts outside the current FOV, a pose for each of those parts with respect to the FOR and the estimated camera pose and renders parts icons with location indicators for parts outside the current FOV over the displayed images of the scene; and an instruction-rendering module that renders the instructions for the use of those parts.