Compact catadioptric projector

A compact projector for eyewear including a refractive lens to achromatize an image and to reduce the size of the projector. The compact projector includes two total internal reflection (TIR) prisms, a polarizing beamsplitter, a quarter-wave plate, and the refractive lens combining refractive and reflective power, referred to as catadioptric. In one example, a light source generates light that is directed through a collector, into a first TIR prism, to the polarizing beamsplitter, and to a display panel. The display panel modulates the light and creates an image. The image is directed through the beamsplitter, into a second TIR prism, through a quarter-wave plate, and then to the refractive lens. The refractive lens reflects the image back into the second TIR prism, and which image then exits to a waveguide. In a second example, the display generates an illuminated image which is then processed as in the first example.

TECHNICAL FIELD

The present subject matter relates to the field of projectors.

BACKGROUND

Many types of projectors generate an image that is viewable to a user, such as used in an eyewear device.

DETAILED DESCRIPTION

This disclosure is directed to a compact projector for eyewear including a refractive lens to achromatize an image and to reduce the size of the projector. The compact projector includes two total internal reflection (TIR) prisms, a polarizing beamsplitter, a quarter-wave plate, and the refractive lens combining refractive and reflective power, referred to as catadioptric. In one example, a light source generates light that is directed through a collector, into a first TIR prism, to the polarizing beamsplitter, and to a display panel. The display panel modulates the light and creates an image. The image is directed through the beamsplitter, into a second TIR prism, through a quarter-wave plate, and then to the refractive lens. The refractive lens reflects the image back into the second TIR prism, and which image then exits to a waveguide. In a second example, the display generates an illuminated image which is then processed as in the first example.

The term “coupled” as used herein refers to any logical, optical, physical or electrical connection, link or the like by which signals or light produced or supplied by one system element are imparted to another coupled element. Unless described otherwise, coupled elements or devices are not necessarily directly connected to one another and may be separated by intermediate components, elements or communication media that may modify, manipulate or carry the light or signals.

The orientations of the eyewear device, associated components and any complete devices incorporating an eye scanner and camera such as shown in any of the drawings, are given by way of example only, for illustration and discussion purposes. In operation for a particular variable optical processing application, the eyewear device may be oriented in any other direction suitable to the particular application of the eyewear device, for example up, down, sideways, or any other orientation. Also, to the extent used herein, any directional term, such as front, rear, inwards, outwards, towards, left, right, lateral, longitudinal, up, down, upper, lower, top, bottom and side, are used by way of example only, and are not limiting as to direction or orientation of any optic or component of an optic constructed as otherwise described herein.

FIG. 1Ais a side view of an example hardware configuration of an eyewear device100, which includes a right optical assembly180B with an image display180D (FIG. 2A). Eyewear device100includes multiple visible light cameras114A-B (FIG. 7) that form a stereo camera, of which the right visible light camera114B is located on a right temple110B.

The left and right visible light cameras114A-B have an image sensor that is sensitive to the visible light range wavelength. Each of the visible light cameras114A-B have a different frontward facing angle of coverage, for example, visible light camera114B has the depicted angle of coverage111B. The angle of coverage is an angle range which the image sensor of the visible light camera114A-B picks up electromagnetic radiation and generates images. Examples of such visible lights camera114A-B include a high-resolution complementary metal-oxide-semiconductor (CMOS) image sensor and a video graphic array (VGA) camera, such as 640p (e.g., 640×480 pixels for a total of 0.3 megapixels), 720p, or 1080p. Image sensor data from the visible light cameras114A-B are captured along with geolocation data, digitized by an image processor, and stored in a memory.

To provide stereoscopic vision, visible light cameras114A-B may be coupled to an image processor (element912ofFIG. 9) for digital processing along with a timestamp in which the image of the scene is captured. Image processor912includes circuitry to receive signals from the visible light camera114A-B and process those signals from the visible light cameras114A-B into a format suitable for storage in the memory (element934ofFIG. 9). The timestamp can be added by the image processor912or other processor, which controls operation of the visible light cameras114A-B. Visible light cameras114A-B allow the stereo camera to simulate human binocular vision. Stereo cameras provide the ability to reproduce three-dimensional images (element715ofFIG. 7) based on two captured images (elements758A-B ofFIG. 7) from the visible light cameras114A-B, respectively, having the same timestamp. Such three-dimensional images715allow for an immersive life-like experience, e.g., for virtual reality or video gaming. For stereoscopic vision, the pair of images758A-B are generated at a given moment in time—one image for each of the left and right visible light cameras114A-B. When the pair of generated images758A-B from the frontward facing field of view (FOV)111A-B of the left and right visible light cameras114A-B are stitched together (e.g., by the image processor912), depth perception is provided by the optical assembly180A-B.

In an example, a user interface field of view adjustment system includes the eyewear device100. The eyewear device100includes a frame105, a right temple110B extending from a right lateral side170B of the frame105, and a see-through image display180D (FIGS. 2A-B) comprising optical assembly180B to present a graphical user interface to a user. The eyewear device100includes the left visible light camera114A connected to the frame105or the left temple110A to capture a first image of the scene. Eyewear device100further includes the right visible light camera114B connected to the frame105or the right temple110B to capture (e.g., simultaneously with the left visible light camera114A) a second image of the scene which partially overlaps the first image. Although not shown inFIGS. 1A-B, the user interface field of view adjustment system further includes the processor932coupled to the eyewear device100and connected to the visible light cameras114A-B, the memory934accessible to the processor932, and programming in the memory934, for example in the eyewear device100itself or another part of the user interface field of view adjustment system.

Although not shown inFIG. 1A, the eyewear device100also includes a head movement tracker (element109ofFIG. 1B) or an eye movement tracker (element213ofFIG. 2B). Eyewear device100further includes the see-through image displays180C-D of optical assembly180A-B, respectfully, for presenting a sequence of displayed images, and an image display driver (element942ofFIG. 9) coupled to the see-through image displays180C-D of optical assembly180A-B to control the image displays180C-D of optical assembly180A-B to present the sequence of displayed images715, which are described in further detail below. Eyewear device100further includes the memory934and the processor932having access to the image display driver942and the memory934. Eyewear device100further includes programming (element934ofFIG. 9) in the memory. Execution of the programming by the processor932configures the eyewear device100to perform functions, including functions to present, via the see-through image displays180C-D, an initial displayed image of the sequence of displayed images, the initial displayed image having an initial field of view corresponding to an initial head direction or an initial eye gaze direction (element230ofFIG. 5).

Execution of the programming by the processor932further configures the eyewear device100to detect movement of a user of the eyewear device by: (i) tracking, via the head movement tracker (element109ofFIG. 1), a head movement of a head of the user, or (ii) tracking, via an eye movement tracker (element213ofFIG. 2B,FIG. 5), an eye movement of an eye of the user of the eyewear device100. Execution of the programming by the processor932further configures the eyewear device100to determine a field of view adjustment to the initial field of view of the initial displayed image based on the detected movement of the user. The field of view adjustment includes a successive field of view corresponding to a successive head direction or a successive eye direction. Execution of the programming by the processor932further configures the eyewear device100to generate a successive displayed image of the sequence of displayed images based on the field of view adjustment. Execution of the programming by the processor932further configures the eyewear device100to present, via the see-through image displays180C-D of the optical assembly180A-B, the successive displayed images.

FIG. 1Bis a top cross-sectional view of the temple of the eyewear device100ofFIG. 1Adepicting the right visible light camera114B, a head movement tracker109, and a circuit board. Construction and placement of the left visible light camera114A is substantially similar to the right visible light camera114B, except the connections and coupling are on the left lateral side170A. As shown, the eyewear device100includes the right visible light camera114B and a circuit board, which may be a flexible printed circuit board (PCB)140. The left hinge126A connects the left temple110A to a left temple extension125A of the eyewear device100. In some examples, components of the left visible light camera114A, the flexible PCB140, or other electrical connectors or contacts may be located on the left temple extension125A or the left hinge126A. The right hinge126B connects the right temple110B to a right temple extension125B of the eyewear device100. In some examples, components of the right visible light camera114B, the flexible PCB140, or other electrical connectors or contacts may be located on the right temple extension125B or the right hinge126B.

As shown, eyewear device100has a head movement tracker109, which includes, for example, an inertial measurement unit (IMU). An inertial measurement unit is an electronic device that measures and reports a body's specific force, angular rate, and sometimes the magnetic field surrounding the body, using a combination of accelerometers and gyroscopes, sometimes also magnetometers. The inertial measurement unit works by detecting linear acceleration using one or more accelerometers and rotational rate using one or more gyroscopes. Typical configurations of inertial measurement units contain one accelerometer, gyro, and magnetometer per axis for each of the three axes: horizontal axis for left-right movement (X), vertical axis (Y) for top-bottom movement, and depth or distance axis for up-down movement (Z). The accelerometer detects the gravity vector. The magnetometer defines the rotation in the magnetic field (e.g., facing south, north, etc.) like a compass which generates a heading reference. The three accelerometers to detect acceleration along the horizontal, vertical, and depth axis defined above, which can be defined relative to the ground, the eyewear device100, or the user wearing the eyewear device100.

Eyewear device100detects movement of the user of the eyewear device100by tracking, via the head movement tracker109, the head movement of the head of the user. The head movement includes a variation of head direction on a horizontal axis, a vertical axis, or a combination thereof from the initial head direction during presentation of the initial displayed image on the image display. In one example, tracking, via the head movement tracker109, the head movement of the head of the user includes measuring, via the inertial measurement unit109, the initial head direction on the horizontal axis (e.g., X axis), the vertical axis (e.g., Y axis), or the combination thereof (e.g., transverse or diagonal movement). Tracking, via the head movement tracker109, the head movement of the head of the user further includes measuring, via the inertial measurement unit109, a successive head direction on the horizontal axis, the vertical axis, or the combination thereof during presentation of the initial displayed image.

Tracking, via the head movement tracker109, the head movement of the head of the user further includes determining the variation of head direction based on both the initial head direction and the successive head direction. Detecting movement of the user of the eyewear device100further includes in response to tracking, via the head movement tracker109, the head movement of the head of the user, determining that the variation of head direction exceeds a deviation angle threshold on the horizontal axis, the vertical axis, or the combination thereof. The deviation angle threshold is between about 3° to 10°. As used herein, the term “about” when referring to an angle means±10% from the stated amount.

Variation along the horizontal axis slides three-dimensional objects, such as characters, Bitmojis, application icons, etc. in and out of the field of view by, for example, hiding, unhiding, or otherwise adjusting visibility of the three-dimensional object. Variation along the vertical axis, for example, when the user looks upwards, in one example, displays weather information, time of day, date, calendar appointments, etc. In another example, when the user looks downwards on the vertical axis, the eyewear device100may power down.

The right temple110B includes temple body211and a temple cap, with the temple cap omitted in the cross-section ofFIG. 1B. Disposed inside the right temple110B are various interconnected circuit boards, such as PCBs or flexible PCBs, that include controller circuits for right visible light camera114B, microphone(s)130, speaker(s)132, low-power wireless circuitry (e.g., for wireless short-range network communication via Bluetooth™), high-speed wireless circuitry (e.g., for wireless local area network communication via WiFi).

The right visible light camera114B is coupled to or disposed on the flexible PCB240and covered by a visible light camera cover lens, which is aimed through opening(s) formed in the right temple110B. In some examples, the frame105connected to the right temple110B includes the opening(s) for the visible light camera cover lens. The frame105includes a front-facing side configured to face outwards away from the eye of the user. The opening for the visible light camera cover lens is formed on and through the front-facing side. In the example, the right visible light camera114B has an outward facing angle of coverage111B with a line of sight or perspective of the right eye of the user of the eyewear device100. The visible light camera cover lens can also be adhered to an outward facing surface of the right temple110B in which an opening is formed with an outwards facing angle of coverage, but in a different outwards direction. The coupling can also be indirect via intervening components.

Left (first) visible light camera114A is connected to the left see-through image display180C of left optical assembly180A to generate a first background scene of a first successive displayed image. The right (second) visible light camera114B is connected to the right see-through image display180D of right optical assembly180B to generate a second background scene of a second successive displayed image. The first background scene and the second background scene partially overlap to present a three-dimensional observable area of the successive displayed image.

Flexible PCB140is disposed inside the right temple110B and is coupled to one or more other components housed in the right temple110B. Although shown as being formed on the circuit boards of the right temple110B, the right visible light camera114B can be formed on the circuit boards of the left temple110A, the temples125A-B, or frame105.

FIG. 2Ais a rear view of an example hardware configuration of an eyewear device100, which includes an eye scanner213on a frame105, for use in a system for determining an eye position and gaze direction of a wearer/user of the eyewear device100. As shown inFIG. 2A, the eyewear device100is in a form configured for wearing by a user, which are eyeglasses in the example ofFIG. 2A. The eyewear device100can take other forms and may incorporate other types of frameworks, for example, a headgear, a headset, or a helmet.

In the eyeglasses example, eyewear device100includes the frame105which includes the left rim107A connected to the right rim107B via the bridge106adapted for a nose of the user. The left and right rims107A-B include respective apertures175A-B which hold the respective optical element180A-B, such as a lens and the see-through displays180C-D. As used herein, the term lens is meant to cover transparent or translucent pieces of glass or plastic having curved and flat surfaces that cause light to converge/diverge or that cause little or no convergence/divergence.

Although shown as having two optical elements180A-B, the eyewear device100can include other arrangements, such as a single optical element depending on the application or intended user of the eyewear device100. As further shown, eyewear device100includes the left temple110A adjacent the left lateral side170A of the frame105and the right temple110B adjacent the right lateral side170B of the frame105. The temples110A-B may be integrated into the frame105on the respective sides170A-B (as illustrated) or implemented as separate components attached to the frame105on the respective sides170A-B. Alternatively, the temples110A-B may be integrated into temples (not shown) attached to the frame105.

In the example ofFIG. 2A, the eye scanner213includes an infrared emitter115and an infrared camera120. Visible light cameras typically include a blue light filter to block infrared light detection, in an example, the infrared camera120is a visible light camera, such as a low-resolution video graphic array (VGA) camera (e.g., 640×480 pixels for a total of 0.3 megapixels), with the blue filter removed. The infrared emitter115and the infrared camera120are co-located on the frame105, for example, both are shown as connected to the upper portion of the left rim107A. The frame105or one or more of the left and right temples110A-B include a circuit board (not shown) that includes the infrared emitter115and the infrared camera120. The infrared emitter115and the infrared camera120can be connected to the circuit board by soldering, for example.

Other arrangements of the infrared emitter115and infrared camera120can be implemented, including arrangements in which the infrared emitter115and infrared camera120are both on the right rim107B, or in different locations on the frame105, for example, the infrared emitter115is on the left rim107A and the infrared camera120is on the right rim107B. In another example, the infrared emitter115is on the frame105and the infrared camera120is on one of the temples110A-B, or vice versa. The infrared emitter115can be connected essentially anywhere on the frame105, left temple110A, or right temple110B to emit a pattern of infrared light. Similarly, the infrared camera120can be connected essentially anywhere on the frame105, left temple110A, or right temple110B to capture at least one reflection variation in the emitted pattern of infrared light.

The infrared emitter115and infrared camera120are arranged to face inwards towards an eye of the user with a partial or full field of view of the eye in order to identify the respective eye position and gaze direction. For example, the infrared emitter115and infrared camera120are positioned directly in front of the eye, in the upper part of the frame105or in the temples110A-B at either ends of the frame105.

FIG. 2Bis a rear view of an example hardware configuration of another eyewear device200. In this example configuration, the eyewear device200is depicted as including an eye scanner213on a right temple210B. As shown, an infrared emitter215and an infrared camera220are co-located on the right temple210B. It should be understood that the eye scanner213or one or more components of the eye scanner213can be located on the left temple210A and other locations of the eyewear device200, for example, the frame105. The infrared emitter215and infrared camera220are like that ofFIG. 2A, but the eye scanner213can be varied to be sensitive to different light wavelengths as described previously inFIG. 2A.

Similar toFIG. 2A, the eyewear device200includes a frame105which includes a left rim107A which is connected to a right rim107B via a bridge106; and the left and right rims107A-B include respective apertures which hold the respective optical elements180A-B comprising the see-through display180C-D.

FIGS. 2C-Dare rear views of example hardware configurations of the eyewear device100, including two different types of see-through image displays180C-D. In one example, these see-through image displays180C-D of optical assembly180A-B include an integrated image display. As shown inFIG. 2C, the optical assemblies180A-B includes a suitable display matrix180C-D of any suitable type, such as a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, a waveguide display, or any other such display.

The optical assembly180A-B also includes an optical layer or layers176, which can include lenses, optical coatings, prisms, mirrors, waveguides, optical strips, and other optical components in any combination. The optical layers176A-N can include a prism having a suitable size and configuration and including a first surface for receiving light from display matrix and a second surface for emitting light to the eye of the user. The prism of the optical layers176A-N extends over all or at least a portion of the respective apertures175A-B formed in the left and right rims107A-B to permit the user to see the second surface of the prism when the eye of the user is viewing through the corresponding left and right rims107A-B. The first surface of the prism of the optical layers176A-N faces upwardly from the frame105and the display matrix overlies the prism so that photons and light emitted by the display matrix impinge the first surface. The prism is sized and shaped so that the light is refracted within the prism and is directed towards the eye of the user by the second surface of the prism of the optical layers176A-N. In this regard, the second surface of the prism of the optical layers176A-N can be convex to direct the light towards the center of the eye. The prism can optionally be sized and shaped to magnify the image projected by the see-through image displays180C-D, and the light travels through the prism so that the image viewed from the second surface is larger in one or more dimensions than the image emitted from the see-through image displays180C-D.

In another example, the see-through image displays180C-D of optical assembly180A-B include a projection image display as shown inFIG. 2D. The optical assembly180A-B includes a projector150, which may be a three-color projector using a scanning mirror, a galvanometer, a laser projector, or other types of projectors. During operation, an optical source such as a projector150is disposed in or on one of the temples125A-B of the eyewear device100. Optical assembly180A-B includes one or more optical strips155A-N spaced apart across the width of the lens of the optical assembly180A-B or across a depth of the lens between the front surface and the rear surface of the lens. A detailed example of a projector is shown inFIGS. 8A-8J.

As the photons projected by the projector150travel across the lens of the optical assembly180A-B, the photons encounter the optical strips155A-N. When a particular photon encounters a particular optical strip, the photon is either redirected towards the user's eye, or it passes to the next optical strip. A combination of modulation of projector150, and modulation of optical strips, may control specific photons or beams of light. In an example, a processor932(FIG. 9) controls optical strips155A-N by initiating mechanical, acoustic, or electromagnetic signals. Although shown as having two optical assemblies180A-B, the eyewear device100can include other arrangements, such as a single or three optical assemblies, or the optical assembly180A-B may have arranged different arrangement depending on the application or intended user of the eyewear device100.

As further shown inFIGS. 2C-D, eyewear device100includes a left temple110A adjacent the left lateral side170A of the frame105and a right temple110B adjacent the right lateral side170B of the frame105. The temples110A-B may be integrated into the frame105on the respective lateral sides170A-B (as illustrated) or implemented as separate components attached to the frame105on the respective sides170A-B. Alternatively, the temples110A-B may be integrated into temples125A-B attached to the frame105.

In one example, the see-through image displays include the first see-through image display180C and the second see-through image display180D. Eyewear device100includes first and second apertures175A-B which hold the respective first and second optical assembly180A-B. The first optical assembly180A includes the first see-through image display180C (e.g., a display matrix ofFIG. 2Cor optical strips155A-N′ and a projector150A). The second optical assembly180B includes the second see-through image display180D e.g., a display matrix ofFIG. 2Cor optical strips155A-N″ and a projector150B). The successive field of view of the successive displayed image includes an angle of view between about 15° to 30, and more specifically 24°, measured horizontally, vertically, or diagonally. The successive displayed image having the successive field of view represents a combined three-dimensional observable area visible through stitching together of two displayed images presented on the first and second image displays.

As used herein, “an angle of view” describes the angular extent of the field of view associated with the displayed images presented on each of the left and right image displays180C-D of optical assembly180A-B. The “angle of coverage” describes the angle range that a lens of visible light cameras114A-B or infrared camera220can image. Typically, the image circle produced by a lens is large enough to cover the film or sensor completely, possibly including some vignetting (i.e., a reduction of an image's brightness or saturation toward the periphery compared to the image center). If the angle of coverage of the lens does not fill the sensor, the image circle will be visible, typically with strong vignetting toward the edge, and the effective angle of view will be limited to the angle of coverage. The “field of view” is intended to describe the field of observable area which the user of the eyewear device100can see through his or her eyes via the displayed images presented on the left and right image displays180C-D of the optical assembly180A-B. Image display180C of optical assembly180A-B can have a field of view with an angle of coverage between 15° to 30°, for example 24°, and have a resolution of 480×480 pixels.

FIG. 3shows a rear perspective view of the eyewear device ofFIG. 2A. The eyewear device100includes an infrared emitter215, infrared camera220, a frame front330, a frame back335, and a circuit board340. It can be seen inFIG. 3that the upper portion of the left rim of the frame of the eyewear device100includes the frame front330and the frame back335. An opening for the infrared emitter215is formed on the frame back335.

As shown in the encircled cross-section4in the upper middle portion of the left rim of the frame, a circuit board, which is a flexible PCB340, is sandwiched between the frame front330and the frame back335. Also shown in further detail is the attachment of the left temple110A to the left temple325A via the left hinge126A. In some examples, components of the eye movement tracker213, including the infrared emitter215, the flexible PCB340, or other electrical connectors or contacts may be located on the left temple325A or the left hinge126A.

FIG. 4is a cross-sectional view through the infrared emitter215and the frame corresponding to the encircled cross-section4of the eyewear device ofFIG. 3. Multiple layers of the eyewear device100are illustrated in the cross-section ofFIG. 4, as shown the frame includes the frame front330and the frame back335. The flexible PCB340is disposed on the frame front330and connected to the frame back335. The infrared emitter215is disposed on the flexible PCB340and covered by an infrared emitter cover lens445. For example, the infrared emitter215is reflowed to the back of the flexible PCB340. Reflowing attaches the infrared emitter215to contact pad(s) formed on the back of the flexible PCB340by subjecting the flexible PCB340to controlled heat which melts a solder paste to connect the two components. In one example, reflowing is used to surface mount the infrared emitter215on the flexible PCB340and electrically connect the two components. However, it should be understood that through-holes can be used to connect leads from the infrared emitter215to the flexible PCB340via interconnects, for example.

The frame back335includes an infrared emitter opening450for the infrared emitter cover lens445. The infrared emitter opening450is formed on a rear-facing side of the frame back335that is configured to face inwards towards the eye of the user. In the example, the flexible PCB340can be connected to the frame front330via the flexible PCB adhesive460. The infrared emitter cover lens445can be connected to the frame back335via infrared emitter cover lens adhesive455. The coupling can also be indirect via intervening components.

In an example, the processor932utilizes eye tracker213to determine an eye gaze direction230of a wearer's eye234as shown inFIG. 5, and an eye position236of the wearer's eye234within an eyebox as shown inFIG. 6. The eye tracker213is a scanner which uses infrared light illumination (e.g., near-infrared, short-wavelength infrared, mid-wavelength infrared, long-wavelength infrared, or far infrared) to captured image of reflection variations of infrared light from the eye234to determine the gaze direction230of a pupil232of the eye234, and also the eye position236with respect to the see-through display180D.

FIG. 7depicts an example of capturing visible light with cameras114A-B. Visible light is captured by the left visible light camera114A with a round field of view (FOV).111A. A chosen rectangular left raw image758A is used for image processing by image processor912(FIG. 9). Visible light is captured by the right visible light camera114B with a round FOV111B. A rectangular right raw image758B chosen by the image processor912is used for image processing by processor912. Based on processing of the left raw image758A and the right raw image758B, a three-dimensional image715of a three-dimensional scene, referred to hereafter as an immersive image, is generated by processor912and displayed by displays180C and180D and which is viewable by the user.

Augmented reality (AR) devices often employ waveguide combiners in front of the user's eye to create a virtual image that overlays the image of the real world passing through the combiner. Light that makes up the virtual image is injected into the waveguide by an input coupling mechanism, for example a diffraction grating or prism, and propagates through the waveguide via total internal reflection (TIR). The light is then extracted from the waveguide by an output coupling mechanism (e.g. a diffraction grating) and forms a pupil some distance from the waveguide where a user can position their eye to view the virtual image.

Referring toFIG. 8A, to form the virtual image and pupil, a display device and projector are positioned such that the exit pupil of the projector is collocated with the input coupler of the waveguide. The projection lens, in this case, acts as a Fourier transform lens which images the display panel and creates an angular distribution of pixels at the projector exit pupil from the panel's spatial pixels.

Referring toFIG. 8B, when the display panel requires external illumination, as in a Liquid-Crystal-on-Silicon (LCoS) based architecture, a second Fourier transform lens (collector) can be employed to capture the illumination from a light source. The system then can be thought of as two Fourier transform lenses (or two relay lens halves) with a display panel at the aperture stop. The light source is then imaged at the input coupler of the waveguide.

Referring toFIG. 8C, there is shown a compact projector150based on a TIR polarizing beamsplitter prism, and a mirrored lens imaging system which combines refractive and reflective power (often called catadioptric).FIG. 8Cillustrates a projector150using a display panel comprising a LCoS spatial light modulator, although in other examples, different light panels can be used, such as a Digital Light Processing (DLP®) manufactured by Texas Instruments of Dallas, Tex., an Organic Light Emitting Diode (OLED), or micro-Light Emitting Diode (uLED) display as will be described shortly.

FIG. 8Cdepicts the architecture of a compact projector150according to this disclosure using a LCoS display panel802. Emitted light806, such as red, green, and blue (RGB) light, is generated from a light source804, such as a light emitting diode (LED), which is controlled by processor932(FIG. 9). The light806is collected by a collector808, such as a compound parabolic concentrator and collection lens. The light806passes through a linear polarizer810and enters a right face of a lower TIR prism812. The light806totally internally reflects from a lower surface814of the lower TIR prism812and is directed toward an upper surface816of the lower TIR prism812. At the upper surface816of the lower TIR prism812is a polarizing beamsplitter (PBS) element820, such as a thin film deposition or a polymer film material, such as PBS-1000 made by 3M of St. Paul, Minn. The PBS element820reflects light of one linear polarization and transmits light of the orthogonal linear polarization. In this case, the input linear polarizer810is oriented such that the light806entering the lower TIR prism812and reaching the PBS element820is s-polarized (instead of p-polarized). The PBS element820reflects the s-polarized light and directs it to the LCoS panel802which is controlled by processor932.

The LCoS panel802operates as a quarter wave plate (QWP) and mirror when pixels of the LCoS panel802are set to the bright state. In this case, s-polarized light enters the LCoS panel802, is rotated by the QWP and mirror to the orthogonal p-state, and then is directed back to the PBS element820. The PBS element820transmits the p-state light and passes it to an upper TIR prism822. Note that the PBS element820includes two PBS films824sandwiching a linear polarizer826. This sandwich enhances the contrast of the device, but is not necessary for operation, and only one PBS film824may be used.

When the LCoS panel802pixels are set to the dark state, the panel acts as a simple mirror. The s-polarized light is reflected back to the PBS element820. The PBS element820reflects the s-polarized light and sends it back through the lower TIR prism812to the light source804. This s-polarized light never reaches the waveguide and therefore constitutes a dark pixel.

When the LCoS panel802pixels are in the bright state, the p-light from the LCoS panel802passes through the upper TIR prism822and to a quarter-wave plate830. The quarter-wave plate830changes the state of the light806from linear to circular. The quarter-wave plate830can be a stack of retardation films laminated to generate an achromatic quarter-wave (AQW) response to red, green, and blue light. Other forms of AQW fabricated by liquid-crystal polymers can be used as well.

The circularly polarized light806then passes through a refractive lens832. The refractive lens832has a mirror coating834on the backside of the lens832to act as a mirror. This type of refractive lens is often termed a Mangin lens (although Mangin lenses typically are associated with negative lens power in the refractive portion of the element). The circularly polarized light806changes handedness upon reflection at the mirror834, and then travels back to the quarter-wave plate830, gathering geometric optical power as it propagates. This catadioptric lens serves as the projection lens. The circularly polarized light806passes again through the quarter-wave plate830and becomes s-polarized light. The s-polarized light reflects at the PBS element820and is sent to an upper edge836of the upper TIR prism822. The light806then reflects at the upper edge836and is sent out an exit port838being the left face of the upper TIR prism822. Note that the quarter-wave plate830may be laminated to the upper TIR prism822for reduced reflections and higher contrast, or it may be a separate element which enhances image quality by using the high-quality prism surface as the TIR reflector.

Light806exiting the exit port838of the upper TIR prism822may be sent through an additional lens840, such as a field lens (but functioning more as a pupil lens), and then through a circular or linear polarizer841. The system creates a Fourier transform of the LCoS panel802which is collocated with an input coupler842of a waveguide844. The input coupler842may be a small grating on the opposite side of the waveguide844from the prism assembly. The input coupler842could also be a prism attached to the waveguide844which injects light into the waveguide844at an angle such that the light TIR's inside the waveguide844.

A field lens may also be inserted between the lower TIR prism812and LCoS panel802to improve the system modulation transfer function (MTF) and lower image distortion.

This TIR PBS prism assembly and catadioptric image system creates a very compact LCoS projector150suitable for injection of an image into the waveguide844.

The basic architecture described above could also be used with other display panels, with some simplification but potentially some light loss. For example, a uLED display panel emits light directly from its pixels. The uLED display panel can be placed in the same position as the LCoS display panel802, and light can be emitted toward the PBS element820. Light in the p-polarized state would pass through the PBS element820, to the quarter-wave plate830, to the Mangin lens832, and back through the quarter-wave plate830where it becomes s-polarized. The s-polarized light then reflects from the PBS element820, reflects at the upper edge836of the upper TIR prism822, and creates a Fourier transform of the uLED panel pixels at the input coupler. uLED display panels typically emit unpolarized light, so only half of the light emitted would pass through the imaging system. The other half would reflect at the initial interception of the PBS element820, and then be sent away from the PBS element820. For this device, a single PBS layer824might be used and no collector or input linear polarizer is required for operation. An organic LED (OLED) display panel could be used in exactly the same way.

Another option is the use a DLP® display panel made by Texas Instruments of Dallas, Tex. Much like the LCoS display panel, the DLP® display panel is a spatial light modulator and does not emit light from the panel. It requires an external source of light. DLP® display panels require illumination that arrives off-axis to the display panel. The DLP® mirrors tilt in one direction to shift light into the projection lens (a bright pixel), and tilt in the opposite direction to direct light away from the projection lens (a dark pixel). If light is directed off-axis, passes through a linear polarizer into the illumination lower TIR prism812, and add a quarter wave plate between the illumination lower TIR prism812and the DLP® display panel, then the system operates in the same manner as the LCoS system. By polarizing the illumination, the brightness of a typical DLP® projector is reduced by 2.

Referring toFIG. 8D, there is illustrated the lens832comprising a Mangin refractive lens coated on one side as mirror834, with a diffractive element to achromatize the image. The lens832could also be a combination of a single lens and a negative dispersion Liquid Crystal Polymer phase lens to achromatize the image.

FIG. 8Eillustrates lens832including two elements configured to achromatize the image, one a crown glass850and the other a flint glass852.

FIG. 8Fillustrates the two elements850and852configured to achromatize the image and a pupil lens854near the input coupler. The pupil lens854improves the Modulation Transfer Function (MTF) of the imaging system.

FIG. 8Gillustrates the two elements850and852configured to achromatize the image, the pupil lens854near the input coupler, and a field lens856near the LCoS panel802. The addition of the field lens856improves the radial distortion of the imaging system.

FIG. 8Hillustrates an example MTF produced by the system depicted inFIG. 8G. In this example, the Nyquist frequency of the pixelated panel is 166 cycles per millimeter, and all field points are above 0.35 modulation contrast.

FIG. 8Iillustrates a turning prism as an input coupler to the waveguide.

FIG. 8Jillustrates a non-telecentric pupil, panel, and light path through illumination lower TIR prism812. A telecentric pupil is located at a very far position (essentially infinity) from the pixelated panel802. A non-telecentric pupil is located at a finite distance from the panel802. By creating a non-telecentric pupil in the imaging portion of the device, the beam footprint through the illumination lower prism812is reduced, and therefore reduces the size of the illumination lower prism812.

At block862, the light source804is controlled by processor932and generates light806. The light source804generates red, green and blue (RGB) light806at different times, such as alternately or sequentially.

At block864the light806is directed through the collector808and the linear polarizer810, and then into the right face of the lower TIR prism812. The light806then reflects off the lower edge814of the lower TIR prism812, and toward the PBS beamsplitter816. The PBS beamsplitter820is s-polarized (instead of p-polarized). The PBS element820reflects the s-polarized light806and directs it to the LCoS panel802which is controlled by processor932.

At block866the LCoS panel802is controlled by the processor932and modulates the received light806to create an image. The LCoS panel802operates as a quarter wave plate (QWP) and mirror when pixels of the LCoS panel802are set to the bright state. In this case, s-polarized light806enters the LCoS panel802, is rotated by the QWP and mirror to the orthogonal p-state, and then is directed back to the PBS element820. The PBS element820transmits the p-state light and passes it to an upper TIR prism822.

At block868the image from the LCoS panel is directed to through the upper edge836of the upper TIR prism822and through the quarter-wave plate830to the refractive lens832. The refractive lens832achromatizes the image. The refractive lens832has a mirror coating834on the backside of the lens832to act as a mirror, such as a Mangin lens. The quarter-wave plate830changes the state of the light806from linear to circular. The circularly polarized light806changes handedness upon reflection at the mirror834, and then travels back to the quarter-wave plate830, gathering geometric optical power as it propagates. This catadioptric lens serves as the projection lens.

At block870the circularly polarized light806passes again through the quarter-wave plate830and becomes s-polarized light. The s-polarized light reflects at the PBS element820and is sent to the upper edge836of the upper TIR prism822. The light806then reflects at the upper edge836and is sent out the exit port838of the upper TIR prism822, through the input coupler842, and into the waveguide844.

FIG. 9depicts a high-level functional block diagram including example electronic components disposed in eyewear100and200. The illustrated electronic components include the processor932, the memory934, and the see-through image display180C and180D.

Memory934includes instructions for execution by processor932to implement functionality of eyewear100/200, including instructions for processor932to control in the image715. Processor932receives power from battery950and executes the instructions stored in memory934, or integrated with the processor932on-chip, to perform functionality of eyewear100/200, and communicating with external devices via wireless connections.

A user interface adjustment system900includes a wearable device, which is the eyewear device100with an eye movement tracker213(e.g., shown as infrared emitter215and infrared camera220inFIG. 2B). User interface adjustments system900also includes a mobile device990and a server system998connected via various networks. Mobile device990may be a smartphone, tablet, laptop computer, access point, or any other such device capable of connecting with eyewear device100using both a low-power wireless connection925and a high-speed wireless connection937. Mobile device990is connected to server system998and network995. The network995may include any combination of wired and wireless connections.

Eyewear device100includes at least two visible light cameras114A-B (one associated with the left lateral side170A and one associated with the right lateral side170B). Eyewear device100further includes two see-through image displays180C-D of the optical assembly180A-B (one associated with the left lateral side170A and one associated with the right lateral side170B). Eyewear device100also includes image display driver942, image processor912, low-power circuitry920, and high-speed circuitry930. The components shown inFIG. 9for the eyewear device100and200are located on one or more circuit boards, for example a PCB or flexible PCB, in the temples. Alternatively, or additionally, the depicted components can be located in the temples, frames, hinges, or bridge of the eyewear device100and200. Left and right visible light cameras114A-B can include digital camera elements such as a complementary metal-oxide-semiconductor (CMOS) image sensor, charge coupled device, a lens, or any other respective visible or light capturing elements that may be used to capture data, including images of scenes with unknown objects.

Eye movement tracking programming945implements the user interface field of view adjustment instructions, including, to cause the eyewear device100to track, via the eye movement tracker213, the eye movement of the eye of the user of the eyewear device100. Other implemented instructions (functions) cause the eyewear device100and200to determine the FOV adjustment to the initial FOV111A-B based on the detected eye movement of the user corresponding to a successive eye direction. Further implemented instructions generate a successive displayed image of the sequence of displayed images based on the field of view adjustment. The successive displayed image is produced as visible output to the user via the user interface. This visible output appears on the see-through image displays180C-D of optical assembly180A-B, which is driven by image display driver934to present the sequence of displayed images, including the initial displayed image with the initial field of view and the successive displayed image with the successive field of view.

As shown inFIG. 9, high-speed circuitry930includes high-speed processor932, memory934, and high-speed wireless circuitry936. In the example, the image display driver942is coupled to the high-speed circuitry930and operated by the high-speed processor932in order to drive the left and right image displays180C-D of the optical assembly180A-B. High-speed processor932may be any processor capable of managing high-speed communications and operation of any general computing system needed for eyewear device100. High-speed processor932includes processing resources needed for managing high-speed data transfers on high-speed wireless connection937to a wireless local area network (WLAN) using high-speed wireless circuitry936. In certain examples, the high-speed processor932executes an operating system such as a LINUX operating system or other such operating system of the eyewear device100and the operating system is stored in memory934for execution. In addition to any other responsibilities, the high-speed processor932executing a software architecture for the eyewear device100is used to manage data transfers with high-speed wireless circuitry936. In certain examples, high-speed wireless circuitry936is configured to implement Institute of Electrical and Electronic Engineers (IEEE) 802.11 communication standards, also referred to herein as Wi-Fi. In other examples, other high-speed communications standards may be implemented by high-speed wireless circuitry936.

Low-power wireless circuitry924and the high-speed wireless circuitry936of the eyewear device100and200can include short range transceivers (Bluetooth™) and wireless wide, local, or wide area network transceivers (e.g., cellular or WiFi). Mobile device990, including the transceivers communicating via the low-power wireless connection925and high-speed wireless connection937, may be implemented using details of the architecture of the eyewear device100, as can other elements of network995.

Memory934includes any storage device capable of storing various data and applications, including, among other things, color maps, camera data generated by the left and right visible light cameras114A-B and the image processor912, as well as images generated for display by the image display driver942on the see-through image displays180C-D of the optical assembly180A-B. While memory934is shown as integrated with high-speed circuitry930, in other examples, memory934may be an independent standalone element of the eyewear device100. In certain such examples, electrical routing lines may provide a connection through a chip that includes the high-speed processor932from the image processor912or low-power processor922to the memory934. In other examples, the high-speed processor932may manage addressing of memory934such that the low-power processor922will boot the high-speed processor932any time that a read or write operation involving memory934is needed.

Server system998may be one or more computing devices as part of a service or network computing system, for example, that include a processor, a memory, and network communication interface to communicate over the network995with the mobile device990and eyewear device100/200. Eyewear device100and200is connected with a host computer. For example, the eyewear device100is paired with the mobile device990via the high-speed wireless connection937or connected to the server system998via the network995.

Output components of the eyewear device100include visual components, such as the left and right image displays180C-D of optical assembly180A-B as described inFIGS. 2C-D(e.g., a display such as a liquid crystal display (LCD), a plasma display panel (PDP), a light emitting diode (LED) display, a projector, or a waveguide). The image displays180C-D of the optical assembly180A-B are driven by the image display driver942. The output components of the eyewear device100further include acoustic components (e.g., speakers), haptic components (e.g., a vibratory motor), other signal generators, and so forth. The input components of the eyewear device100and200, the mobile device990, and server system998, may include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or other pointing instruments), tactile input components (e.g., a physical button, a touch screen that provides location and force of touches or touch gestures, or other tactile input components), audio input components (e.g., a microphone), and the like.

Eyewear device100may optionally include additional peripheral device elements919. Such peripheral device elements may include ambient light and spectral sensors, biometric sensors, additional sensors, or display elements integrated with eyewear device100. For example, peripheral device elements919may include any I/O components including output components, motion components, position components, or any other such elements described herein. The eyewear device100can take other forms and may incorporate other types of frameworks, for example, a headgear, a headset, or a helmet.

For example, the biometric components of the user interface field of view adjustment900include components to detect expressions (e.g., hand expressions, facial expressions, vocal expressions, body gestures, or eye tracking), measure biosignals (e.g., blood pressure, heart rate, body temperature, perspiration, or brain waves), identify a person (e.g., voice identification, retinal identification, facial identification, fingerprint identification, or electroencephalogram based identification), and the like. The motion components include acceleration sensor components (e.g., accelerometer), gravitation sensor components, rotation sensor components (e.g., gyroscope), and so forth. The position components include location sensor components to generate location coordinates (e.g., a Global Positioning System (GPS) receiver component), WiFi or Bluetooth™ transceivers to generate positioning system coordinates, altitude sensor components (e.g., altimeters or barometers that detect air pressure from which altitude may be derived), orientation sensor components (e.g., magnetometers), and the like. Such positioning system coordinates can also be received over wireless connections925and937from the mobile device990via the low-power wireless circuitry924or high-speed wireless circuitry936.

According to some examples, an “application” or “applications” are program(s) that execute functions defined in the programs. Various programming languages can be employed to create one or more of the applications, structured in a variety of manners, such as object-oriented programming languages (e.g., Objective-C, Java, or C++) or procedural programming languages (e.g., C or assembly language). In a specific example, a third party application (e.g., an application developed using the ANDROID™ or IOS™ software development kit (SDK) by an entity other than the vendor of the particular platform) may be mobile software running on a mobile operating system such as IOS™, ANDROID™ WINDOWS® Phone, or another mobile operating systems. In this example, the third-party application can invoke API calls provided by the operating system to facilitate functionality described herein.

While the foregoing has described what are considered to be the best mode and other examples, it is understood that various modifications may be made therein and that the subject matter disclosed herein may be implemented in various forms and examples, and that they may be applied in numerous applications, only some of which have been described herein. It is intended by the following claims to claim any and all modifications and variations that fall within the true scope of the present concepts.