Patent Description:
According to aspects of the present invention, there is provided a device as defined in the accompanying claims.

A near-eye optical display system utilizes a compact display engine that couples image light from an imager to a waveguide-based display having diffractive optical elements (DOEs) that provide exit pupil expansion in two directions. The display engine comprises a pair of single axis MEMS (micro electro mechanical system) scanners that are configured to reflect the image light through horizontal and vertical scan axes of the display system's field of view (FOV) using raster scanning. The MEMS scanners are arranged with their axes of rotation at substantially right angles to each other and operate with respective quarter wave retarder plates and a polarizing beam splitter (PBS) to couple the image light into an in-coupling DOE in the waveguide display without the need for additional optical elements such as lenses or relay systems. The display engine can thus be compact and lightweight which are typically desirable characteristics in many applications, particularly in wearable systems such as head mounted display (HMD) devices that can support mixed-reality and virtual-reality imaging applications.

In a first illustrative embodiment of the display engine not according to the claimed invention, two MEMS scanners - a slow scan MEMS scanner (i.e., configured to sweep along one direction of the FOV) and a fast scan MEMS scanner (i.e., configured to sweep along the other direction of the FOV) - are located on opposite, top and bottom faces of a PBS cube and adjacent to two respective quarter wave retarder plates. The PBS cube comprises two right-angled prisms that are joined along their hypotenuse surfaces and include a polarization-sensitive beam splitter interface, such as a dielectric coating. The two faces of each right-angle prism opposite the hypotenuse may be used as entrance faces for impinging light on the PBS cube or as exit faces for light that is transmitted or reflected from the cube.

The beam splitter interface of the PBS cube reflects light having a first state of polarization while transmitting light having a second polarization state that is orthogonal to the first state. For example, the first and second polarization states can be s- and p-polarization states (or vice-versa). An imager which may comprise, for example, one or more lasers generates image light such as holographic virtual images used in mixed- or virtual-reality applications. The image light is polarized in the first polarization state and enters an entrance face of the PBS cube. The in-coupling DOE is located on the opposite exit face of the PBS cube. The PBS cube reflects the incident image light upwards (i.e., orthogonally to the incident image light) to the fast scan MEMS scanner located at the top face of the cube where the image light is reflected downwards back to the PBS cube.

The image light reflected by the PBS cube makes two passes through the quarter wave plate located at the top face of the PBS cube - one pass upward to the fast scan MEMS scanner and one pass downward when reflected from the fast scan MEMS scanner - and thus changes to a second polarization state. The image light in the second polarization state passes through the PBS cube, without being reflected at the interface, to the slow scan MEMS scanner at the bottom face of the PBS cube. The slow scan MEMS scanner reflects the image light upwards back into the PBS cube. The state of polarization changes back to the first state as the image light makes two passes through the quarter wave plate at the bottom face of the PBS cube. The PBS cube reflects the image light in the first polarization state out of the cube at the exit face and into the in-coupling DOE to thereby couple the image light from the imager into the waveguide display.

In a second illustrative embodiment of the display engine not according to the claimed invention, a fast scan MEMS scanner and quarter wave plate are located at the top face of a PBS cube opposite the entrance face on the bottom of the cube. A slow scan MEMS scanner and quarter wave plate are located on a face of the PBS cube that is opposite the exit face at which an in-coupling DOE is located. Image light thus enters the PBS cube in a direction that is parallel to the plane of the in-coupling DOE (as compared to the first embodiment in which the image light enters the PBS cube in a direction that is orthogonal to the plane of the in-coupling DOE).

In the second illustrative embodiment of the display engine, the image light incident on the bottom entrance face of the PBS cube is initially polarized in the second state so that it is not subject to reflection at the beam splitter interface and thereby is transmitted to the quarter wave plate and fast scan MEMS scanner located at the opposite top face. As the image light reflected downward from the fast scan MEMS scanner has made two passes through the top quarter wave plate, it is changed to the first polarization state (i.e., orthogonal to the second polarization state).

The PBS cube reflects the image light to the slow scan MEMS scanner and quarter wave plate which are located on the face opposite the exit face of the PBS cube. The slow scan MEMS scanner reflects the image light back towards the PBS cube. Since the reflected image light has made two passes through the quarter wave plate, it is changed back to the second polarization state and thereby passes through the PBS cube without being reflected at the beam splitter interface and is coupled into the waveguide display by the in-coupling DOE.

In a third illustrative embodiment of the display engine according to the claimed invention, the in-coupling DOE is configured with polarization sensitivity to the first polarization state, for example, using a Bragg grating and/or two-dimensional grating structures. The in-coupling DOE is located between the exit face of a PBS cube and a slow scan MEMS scanner and quarter wave plate. A fast scan MEMS scanner and quarter wave plate are located at a face of the PBS cube that is opposite to the exit face. Image light in a first polarization state and having a direction of propagation that is parallel to the plane of the in-coupling DOE is incident on the entrance face of the PBS cube (e.g., the top or bottom face). The beam splitter interface of the PBS cube is configured with sensitivity to the first polarization state and reflects the incident image light horizontally to the fast scan MEMS scanner which reflects the image light back to the PBS cube.

The polarization state of the image light reflected from the fast scan MEMS scanner is changed to the second polarization state (i.e., orthogonal to the first polarization state) because it has made two passes through the quarter wave plate that is adjacent to the fast scan MEMS scanner. The image light in the second polarization state is transmitted, through the PBS cube without reflection at the beam splitter interface, to the in-coupling DOE. The in-coupling DOE is sensitive to a first polarization state and therefore passes the image light in the second polarization state to the quarter wave plate and slow scan MEMS scanner without in-coupling the light to the waveguide display. The slow scan MEMS scanner reflects the image light back to the in-coupling DOE. As the light reflected from the slow scan MEMS scanner has made two passes through the quarter wave plate, its state is changed to the first polarization state. The in-coupling DOE, being sensitive to the first polarization state, couples the image light into the waveguide display.

Advantageously, the display engine in each embodiment may be configured using a small form factor PBS cube that places the fast and slow MEMS scanners proximate to each other so that the scanning mirror footprints can be minimized for a given FOV to reduce the overall size and weight of the display engine. In addition, providing exit pupil expansion in the waveguide display enables the display engine to directly couple the output of the imager into the waveguide display without the need for re-imaging on a screen that may manifest visible laser speckle.

Like reference numerals indicate like elements in the drawings. Elements are not drawn to scale unless otherwise indicated.

<FIG> shows a block diagram of an illustrative near-eye optical display system <NUM> which may incorporate a combination of diffractive optical elements (DOEs) that provide in-coupling of incident light into a waveguide, exit pupil expansion in two directions, and out-coupling of light out of the waveguide. Near-eye optical display systems are often used, for example, in head mounted display (HMD) devices in industrial, commercial, and consumer applications. Other devices and systems may also use near-eye display systems, as described below. The near-eye optical display system <NUM> is an example that is used to provide context and illustrate various features and aspects of the present compact display engine with MEMS scanners.

System <NUM> may include one or more imagers (representatively indicated by reference numeral <NUM>) that work with an optical system <NUM> to deliver images as a virtual display to a user's eye <NUM>. The imager <NUM> may include, for example, RGB (red, green, blue) light emitting diodes (LEDs), LCOS (liquid crystal on silicon) devices, OLED (organic light emitting diode) arrays, lasers, laser diodes, or any other suitable displays or micro-displays operating in transmission, reflection, or emission. The optical system <NUM> can typically include a display engine <NUM>, pupil forming optics <NUM>, and one or more waveguides <NUM>. The imager <NUM> may include or incorporate an illumination unit and/or light engine (not shown) that may be configured to provide illumination in a range of wavelengths and intensities in some implementations.

In a near-eye optical display system the imager does not actually shine the images on a surface such as a glass lens to create the visual display for the user. This is not feasible because the human eye cannot focus on something that is that close. Rather than create a visible image on a surface, the near-eye optical display system <NUM> uses the pupil forming optics <NUM> to form a pupil and the eye <NUM> acts as the last element in the optical chain and converts the light from the pupil into an image on the eye's retina as a virtual display.

The waveguide <NUM> facilitates light transmission between the imager and the eye. One or more waveguides can be utilized in the near-eye optical display system because they are transparent and because they are generally small and lightweight (which is desirable in applications such as HMD devices where size and weight is generally sought to be minimized for reasons of performance and user comfort). For example, the waveguide <NUM> can enable the imager <NUM> to be located out of the way, for example, on the side of the user's head or near the forehead, leaving only a relatively small, light, and transparent waveguide optical element in front of the eyes. In one implementation, the waveguide <NUM> operates using a principle of total internal reflection, as shown in <FIG>, so that light can be coupled among the various optical elements in the system <NUM>.

<FIG> shows a view of an illustrative exit pupil expander (EPE) <NUM>. EPE <NUM> receives an input optical beam from the imager <NUM> and the display engine <NUM> as an entrance pupil to produce one or more output optical beams with expanded exit pupil in one or two directions relative to the input (in general, the input may include more than one optical beam which may be produced by separate sources). The display engine <NUM> is described in more detail below and replaces magnifying and/or collimating optics that are typically used in conventional display systems. The expanded exit pupil typically facilitates a virtual display to be sufficiently sized to meet the various design requirements such as image resolution, field of view, and the like of a given optical system while enabling the imager and associated components to be relatively light and compact.

The EPE <NUM> is configured, in this illustrative example, to provide binocular operation for both the left and right eyes which may support stereoscopic viewing. Components that may be utilized for stereoscopic operation such as scanning mirrors, lenses, filters, beam splitters, MEMS devices, or the like are not shown in <FIG> for sake of clarity in exposition. The EPE <NUM> utilizes two out-coupling gratings, <NUM>L and <NUM>R that are supported on a waveguide <NUM> and a central in-coupling grating <NUM>. The in-coupling and out-coupling gratings may be configured using multiple DOEs, and may further include one or more intermediate DOEs as described below. While the EPE <NUM> is depicted as having a planar configuration, other shapes may also be utilized including, for example, curved or partially spherical shapes, in which case the gratings disposed thereon are non-co-planar.

As shown in <FIG>, the EPE <NUM> may be configured to provide an expanded exit pupil in two directions (i.e., along each of a first and second coordinate axis). As shown, the exit pupil is expanded in both the vertical and horizontal directions. It may be understood that the terms "left," "right," "up," "down," "direction," "horizontal," and "vertical" are used primarily to establish relative orientations in the illustrative examples shown and described herein for ease of description. These terms may be intuitive for a usage scenario in which the user of the near-eye optical display device is upright and forward facing, but less intuitive for other usage scenarios. The listed terms are not to be construed to limit the scope of the configurations (and usage scenarios therein) of near-eye optical display features utilized in the present arrangement. The entrance pupil to the EPE <NUM> at the in-coupling grating <NUM> is generally described in terms of field of view (FOV), for example, using horizontal FOV, vertical FOV, or diagonal FOV as shown in <FIG>.

<FIG> shows an illustrative example of a visor <NUM> that incorporates an internal near-eye optical display system that is used in a head mounted display (HMD) device <NUM> application worn by a user <NUM>. The visor <NUM>, in this example, is sealed to protect the internal near-eye optical display system. The visor <NUM> typically interfaces with other components of the HMD device <NUM> such as head mounting/retention systems and other subsystems including sensors, power management, controllers, etc., as illustratively described in conjunction with <FIG> and <FIG>. Suitable interface elements (not shown) including snaps, bosses, screws and other fasteners, etc. may also be incorporated into the visor <NUM>.

The visor <NUM> includes see-through front and rear shields, <NUM> and <NUM> respectively, that can be molded using transparent materials to facilitate unobstructed vision to the optical displays and the surrounding real world environment. Treatments may be applied to the front and rear shields such as tinting, mirroring, anti-reflective, anti-fog, and other coatings, and various colors and finishes may also be utilized. The front and rear shields are affixed to a chassis <NUM> shown in the disassembled view in <FIG>.

The sealed visor <NUM> can physically protect sensitive internal components, including a near-eye optical display system <NUM> (shown in <FIG>), when the HMD device is used in operation and during normal handling for cleaning and the like. The near-eye optical display system <NUM> includes left and right waveguide displays <NUM> and <NUM> that respectively provide virtual world images to the user's left and right eyes for mixed- and/or virtual-reality applications. The visor <NUM> can also protect the near-eye optical display system <NUM> from environmental elements and damage should the HMD device be dropped or bumped, impacted, etc..

As shown in <FIG>, the rear shield <NUM> is configured in an ergonomically suitable form to interface with the user's nose, and nose pads and/or other comfort features can be included (e.g., molded-in and/or added-on as discrete components). The sealed visor <NUM> can also incorporate some level of optical diopter curvature (i.e., eye prescription) within the molded shields in some cases.

<FIG> shows an illustrative waveguide display <NUM> having four DOEs that may be used with, or incorporated as a part of, a see-through waveguide <NUM> to provide in-coupling, expansion of the exit pupil in two directions, and out-coupling. The waveguide display <NUM> may be utilized in an exit pupil expander that is included in the near eye display system <NUM> (<FIG>) to provide virtual world images to one of the user's eyes. Each DOE is an optical element comprising a periodic structure that can modulate various properties of light in a periodic pattern such as the direction of optical axis, optical path length, and the like. The structure can be periodic in one dimension such as one-dimensional (1D) grating and/or be periodic in two dimensions such as two-dimensional (2D) grating, as described in more detail below in the text accompanying <FIG>.

The waveguide display <NUM> includes an in-coupling DOE <NUM>, an out-coupling DOE <NUM>, and left and right intermediate DOEs <NUM> and <NUM> that couple light between the in-coupling and out-coupling DOEs. The in-coupling DOE <NUM> is configured to couple image light comprising one or more imaging beams from an imager <NUM> (<FIG>) into the waveguide. The intermediate DOEs <NUM> and <NUM> expand the exit pupil in a first direction along a first coordinate axis, and the out-coupling DOE <NUM> expands the exit pupil in a second direction along a second coordinate axis and couples light out of the waveguide to the user's eye. The angle ρ is a rotation angle between the periodic lines of the in-coupling DOE <NUM> and the right intermediate DOE <NUM> as shown. As the light propagates in the right intermediate DOE <NUM> (horizontally from left to right in the drawing), it is also diffracted (in the downward direction) to the out-coupling DOE <NUM>. As the light propagates in the left intermediate DOE <NUM> (horizontally from right to left in the drawing), it is also diffracted (in the downward direction) to the out-coupling DOE <NUM>.

While four DOEs are shown in this illustrative example in which a central in-coupling DOE is disposed between the two intermediate DOEs above the out-coupling DOE, in some implementations a single intermediate DOE may be utilized. In this case, the in-coupling DOE may be laterally positioned within the waveguide along a side of the intermediate DOE. The single intermediate DOE and out-coupling DOE in the arrangement of three DOEs are configured to provide exit pupil expansion in two directions in a similar manner to the four DOE arrangement. It may be appreciated that other numbers and arrangements of DOEs may be utilized to meet the needs of a particular implementation.

The grating features used in the DOEs in the waveguide display <NUM> (<FIG>) can take various suitable forms. For example, <FIG> shows a profile of straight (i.e., non-slanted) grating features <NUM> (referred to as grating bars, grating lines, or simply "gratings"), that are formed in a substrate <NUM> in 1D and 2D grating arrangements. By comparison, <FIG> shows grating features <NUM> formed in a substrate <NUM> that have an asymmetric profile. That is, the gratings may be slanted (i.e., non-orthogonal) relative to a plane of the waveguide. In implementations where the waveguide is non-planar, then the gratings may be slanted relative to a direction of light propagation in the waveguide. Asymmetric grating profiles can also be implemented using blazed gratings, or echelette gratings, in which grooves are formed to create grating features with asymmetric triangular or sawtooth profiles, for example to support polarization-sensitive Bragg gratings and other structures. In <FIG>, the grating period is represented by d, the grating height by h, the bar width by c, and the filling factor by f, where f =c/d. The slanted gratings in <FIG> may be described by slant angles α<NUM> and α<NUM>.

<FIG> show various illustrative 2D diffraction gratings which may be utilized to impart polarization sensitivity to the in-coupling DOE as discussed in the illustrative embodiment shown in <FIG>. The 2D gratings in the <FIG> are intended to be illustrative and not limiting, and it is contemplated that variations from the 2D gratings shown may also be utilized. Gratings may include symmetric and/or asymmetric features including slanted gratings (i.e., gratings having walls that are non-orthogonal according to one or more predetermined angles to a plane of the waveguide) and blazed gratings (i.e., gratings having asymmetric triangular or sawtooth profiles) in some cases. Various suitable surface relief contours, filling factors, grating periods, and grating dimensions can also be utilized to meet the needs of a particular implementation.

<FIG> shows a 2D grating <NUM> that includes quadrangular elements that project from a substrate. The quadrangular elements can also be configured to be asymmetric such as being slanted or blazed. Non-quadrangular three-dimensional geometries (both symmetric and asymmetric) may also be utilized for a 2D grating including, for example, cylindrical elements, polygonal elements, elliptical elements, or the like. For example, <FIG> shows a 2D grating <NUM> that includes pyramidal elements, and <FIG> shows a 2D grating <NUM> that includes elements that have a blazed profile in each of the x and z directions. Gratings may also have elements with curved profiles, as shown in the illustrative 2D grating <NUM> in <FIG>.

<FIG> show respective simplified side and top views of an illustrative MEMS (Micro Electro Mechanical System) device <NUM> that includes a scanning plate <NUM> configured to scan one or more beams that comprise image light for virtual images. The MEMS device <NUM> is often referred to as a MEMS scanner or beam deflector. The scanning plate <NUM> comprises a reflective surface (e.g., mirror) that is used to scan an impinging beam over an FOV which is movably suspended to one or more structures (not shown) in a MEMS device using lateral torsional flexures <NUM> and <NUM>, or other suitable arrangements such as bending flexures. The reflective surface may include a plated reflective metal such as gold or aluminum, a dielectric stack, bare silicon, or other materials depending upon wavelength and other design criteria. The scanning plate <NUM> may be configured with a rectangular footprint as shown in <FIG>, although circular or oval footprints may also be utilized in some applications as indicated by the dashed line <NUM>.

Various actuation technologies (not shown in the drawings) for MEMS scanners may be utilized depending on the needs of a particular implementation. Electrocapacitive drive scanners include both rear drive pad and comb drive architectures. Magnetic drive scanners include moving coil and moving magnet types. Other technologies include thermal, piezoelectric, and impact motor drives. Electrocapacitive drive systems may be referred to as electrostatic and bending flexures may be referred to as cantilever arms. MEMS scanners may be operated non-resonantly, and resonantly in some cases which may reduce power consumption.

In this example, the MEMS scanners are configured as single axis (i.e., 1D) scanners that are operated in pairs to provide 2D scanning whereby the axes of rotation, indicated by reference numeral <NUM>, are positioned to be at substantially right angles to each other. One MEMS scanner in the pair is operated to perform a fast scan, while the other is operated to perform a slow scan. Typically, the fast scan MEMS scanner sweeps back and forth horizontally across the FOV while the slow scan MEMS scanner indexes down the FOV by one or two lines. Such systems may be termed progressive scan systems in which the beams of image light may be scanned unidirectionally or bidirectionally depending upon the desired resolution, frame rate, and scanner capabilities.

The fast scan MEMS scanner generally operates at a relatively high scan rate while the slow scan MEMS scanner operates at a scan rate equal to the video frame rate. In some applications, the fast scan MEMS scanner operates resonantly while the slow scan MEMS scanner provides a substantially sawtooth pattern, scanning progressively down the frame for a portion of a frame and then flying back to the top of the frame to start over. In other applications, interleaved sawtooth scanning, triangular wave scanning, sinusoidal scanning, and other waveforms are used to drive one or both axes.

Depending on application requirements, the fast scan direction can be horizontal (rotating about a vertical scan axis) and a slow scan direction can be vertical (rotating about a horizontal scan axis). However, such convention is not limiting and some embodiments of the present compact display engine with MEMS scanners may be implemented with fast and slow scans in the vertical and horizontal directions, respectively, as described below, as well as other directions.

<FIG> shows an illustrative polarizing beam splitter (PBS) cube <NUM> that comprises two right angle prisms <NUM> and <NUM> which are joined at a planar interface <NUM> defined by each prism's hypotenuse. The PBS cube is configured as a hexahedron in typical implementations with square faces, however other configurations such as rhomboid prisms (i.e., lateral displacement beam splitters) can also be utilized in some implementations. Other beam splitting devices may be alternatively utilized including, for example, plate beam splitters, wire grid beam splitters, diffraction grating beam splitters, and other suitable beam splitters.

The interface between the prisms (referred to here as a "beam splitter interface") is configured to be polarization-sensitive using, for example, a dielectric beam splitter coating that can reflect and transmit a portion of an incident light beam. When an incoming randomly polarized or unpolarized beam <NUM> is incident on the entrance face <NUM>, the PBS cube splits the beam into two orthogonal, linearly polarized components including an s-polarized component and a p-polarized component (s-polarized light is also referred to as TE (transverse electric), and p-polarized as TM (transverse magnetic)). S-polarized light <NUM> is reflected at a <NUM>-degree angle with respect to the incident beam <NUM> while p-polarized light <NUM> is transmitted through the PBS cube without being altered. That is, the PBS cube provides a <NUM>-degree separation between the reflected and transmitted beams. In some implementations, one or more of the four entrance/exit faces of the PBS cube may be coated with an antireflection coating to minimize back reflections.

<FIG> show an illustrative quarter wave plate <NUM> that comprises a slab of birefringent material having thickness, d. Quarter wave plates are also referred to as birefringent wave plates or retarders. The quarter wave plate is configured so that the optical axis is parallel to the front and rear faces and includes fast and slow axes which are orthogonal to each other as well as to the light beam propagation direction.

The velocities of extraordinary and ordinary rays through the birefringent materials vary inversely with their refractive indices. The difference in velocities gives rise to a phase difference when the two rays recombine. In the case of an incident linearly polarized beam this is given by α =2πd(ne-no)/λ where α is the phase difference; ne, and no are refractive indices of extraordinary and ordinary rays respectively; and λ is wavelength. Accordingly, at any specific wavelength the phase difference is governed by the thickness, d, of the wave plate <NUM>. The quarter wave plate provides a phase difference of π/<NUM> which corresponds to a propagation shift of X/<NUM> (thus the name "quarter wave plate").

A linearly polarized beam <NUM> incident on the front face <NUM> of the quarter wave plate <NUM> changes to a circularly polarized beam <NUM> upon exit from the opposite back face <NUM>. In this example, the input beam <NUM> is s-polarized in the horizontal plane and the fast and slow axes of the quarter wave plate are positioned diagonally in the plate. That is, the plane of the incident light makes a <NUM>-degree angle with the optical axis of the quarter wave plate.

As shown in <FIG>, when the exiting beam impinges on a reflective surface <NUM> (such as the reflective scanning plate in a MEMS scanner), it is reflected in a backward propagating circularly polarized beam <NUM> towards the quarter wave plate <NUM>. The backward propagating circularly polarized beam <NUM> impinges on the back face of the quarter wave plate where it changes to a linearly polarized beam <NUM>. Beam <NUM> exits from the front face with an opposite polarization state to that of the impinging beam on the front face. In this example, the exiting beam <NUM> is p-polarized. Thus, when a linearly polarized beam makes two passes through the quarter wave plate - one pass in the forward propagating direction and the second in the backward propagating direction - its state of polarization changes to the orthogonal state of polarization (e.g., from s-polarized to p-polarized or vice versa).

<FIG> shows a first illustrative embodiment not according to the claimed invention of a compact display engine <NUM> that comprises a pair of MEMS scanners, including a fast scan MEMS scanner <NUM> and a slow scan MEMS scanner <NUM>, and a respective pair of adjacent quarter wave plates, including a top quarter wave plate <NUM> and a bottom quarter wave plate <NUM> that are disposed about a PBS cube <NUM>. The PBS cube is configured to reflect linearly polarized light in a first polarization state and transmit linearly polarized light in a second polarization state that is orthogonal to the first state. For example, the first polarization state may be s-polarized and the second polarization state may be p-polarized.

The display engine <NUM> is located adjacent to the in-coupling DOE <NUM> of a waveguide display (e.g., waveguide display <NUM> shown in <FIG> and described in the accompanying text). In this embodiment, the fast scan MEMS scanner is located at the top face of the PBS cube <NUM> and the slow scan MEMS scanner is located at the bottom face of the PBS cube. The MEMS scanners are oriented with orthogonal scanning axes, as shown.

In this illustrative embodiment and the others described below, it may be appreciated that the fast scan and slow scan MEMS scanners are arranged in the display engine in a manner that produces image light that is coupled into the waveguide display with an FOV that is rotated compared to some conventional raster scanning arrangements. That is, the vertical and horizontal axes are switched compared to some conventional systems. Compensation for such FOV rotation in the waveguide display may be provided, for example, by corresponding changes in the virtual images as they are produced prior to rendering and display.

As shown in <FIG>, image light <NUM> from an imager (e.g., imager <NUM> in <FIG>) impinges on an entrance face <NUM> of the PBS cube <NUM>. The image light in this example not according to the claimed invention, comprises laser light using one or more beams which may cover the expanse of a given FOV. The image light is polarized in the first state. In some implementations, the light from the laser is inherently linearly polarized and the polarization is controlled through alignment of the imager with respect to the display engine, or using a polarization rotator or other suitable device. In other implementations, the image light may be transmitted through a polarizing filter.

The beam splitter interface <NUM> of the PBS cube <NUM> is oriented at a <NUM>-degree angle in a vertical plane to the direction of propagation of the image light. The beam splitter interface thus reflects the incident image light upwards (i.e., orthogonally to the incident image light) to the top quarter wave plate <NUM> and fast scan MEMS scanner <NUM>.

As shown in <FIG>, the fast scan MEMS scanner reflects the image light downwards back to the PBS cube. The image light reflected by the beam splitter interface <NUM> makes two passes through the top quarter wave plate <NUM> - one pass upward to the fast scan MEMS scanner <NUM> and one pass downward when reflected from the fast scan MEMS scanner - and thus changes to a second polarization state. The image light in the second polarization state passes through the PBS cube <NUM>, without being reflected at the beam splitter interface <NUM>, to the slow scan MEMS scanner <NUM> at the bottom face of the PBS cube.

The slow scan MEMS scanner <NUM> reflects the image light upwards back to the PBS cube. The state of polarization changes back to the first state as the image light makes two passes through the bottom quarter wave plate <NUM>. The beam splitter interface <NUM> reflects the image light in the first polarization state out of the cube at the exit face <NUM> and into the in-coupling DOE <NUM> to thereby couple the image light from the imager into the waveguide display.

In the first illustrative embodiment not according to the claimed invention, as shown in <FIG>, the image light propagates orthogonally to the plane of the in-coupling DOE <NUM>. While this arrangement may be advantageous in some applications to optimize packaging and other design considerations, it can present a risk that high intensity laser light may enter the in-coupling DOE or other portions of the waveguide display. Such stray image light in the waveguide display can result in suboptimal image quality in some cases and may result in less efficient energy utilization in a given device.

In a second illustrative embodiment of a display engine <NUM> not according to the claimed invention, shown in <FIG>, image light <NUM> in a second polarization state is incident on an entrance face <NUM> at the bottom of a PBS cube <NUM>. The image light thus enters the PBS cube in a direction that is parallel to the plane of the in-coupling DOE <NUM>. As in the first embodiment, the beam splitter interface <NUM> of the PBS cube in this second embodiment is configured to be sensitive to the first polarization state. Accordingly, the image light is not reflected at the beam splitter interface, and instead is transmitted through the PBS cube to the top quarter wave plate <NUM> and fast scan MEMS scanner <NUM> located at the top face of the cube opposite the entrance face.

As shown in <FIG>, the fast scan MEMS scanner <NUM> reflects the light downward through the PBS cube <NUM>. As the reflected image light has made two passes through the top quarter wave plate <NUM> - one pass upwards and one pass downwards - it is changed to the first polarization state (i.e., orthogonal to the second polarization state). The beam splitter interface <NUM>, being sensitive to the first polarization state, reflects the image light horizontally to the slow scan MEMS scanner <NUM> and side quarter wave plate <NUM> on the side face <NUM> of the PBS cube opposite the exit face <NUM>.

The slow scan MEMS scanner <NUM> reflects the image light back towards the PBS cube. As the reflected image light has made two passes through the side quarter wave plate <NUM>, it is changed back to the second polarization state and thereby passes through the PBS cube without being reflected at the beam splitter interface <NUM>. The image light exiting the PBS cube at the exit face <NUM> is coupled into the waveguide display by the in-coupling DOE <NUM>.

<FIG>, and <FIG> show a third illustrative embodiment of a display engine <NUM> according to the claimed invention, which may provide for additional packaging flexibility and/or space savings in some implementations. In this embodiment, the in-coupling DOE <NUM> is configured with polarization sensitivity to the first polarization state, for example, using a Bragg grating and/or two-dimensional grating structure, as described above in the text accompanying <FIG>. The in-coupling DOE <NUM> is located between the exit face <NUM> of a PBS cube <NUM> and a slow scan MEMS scanner <NUM> and associated quarter wave plate <NUM>. A fast scan MEMS scanner <NUM> and associated quarter wave plate <NUM> are located at a side face <NUM> of the PBS cube that is opposite to the exit face <NUM>.

Image light <NUM> in a first polarization state propagates in a direction that is parallel to the plane of the in-coupling DOE <NUM>. In this embodiment, the image light enters the top face <NUM> of the PBS cube <NUM>. The beam splitter interface <NUM> of the PBS cube is configured with sensitivity to the first polarization state and therefore reflects the image light horizontally to the side quarter wave plate <NUM> and fast scan MEMS scanner <NUM>.

As shown in <FIG>, the fast scan MEMS scanner <NUM> reflects the image light back to the PBS cube <NUM>. The polarization state of the image light reflected from the fast scan MEMS scanner <NUM> is changed to the second polarization state (i.e., orthogonal to the first polarization state) because it has made two passes through the side quarter wave plate <NUM>. As the in-coupling DOE <NUM> is sensitive to the first polarization state, the reflected image light in the second polarization state is transmitted, through the in-coupling DOE without being in-coupled to the waveguide, to the quarter wave plate <NUM> and slow scan MEMS scanner <NUM> on the exit side of the in-coupling DOE <NUM>.

As shown in the enlarged view in <FIG>, the slow scan MEMS scanner <NUM> reflects the image light back to in-coupling DOE <NUM>. As the light reflected from the slow scan MEMS scanner has made two passes through the quarter wave plate <NUM>, its state is changed to the first polarization state. The in-coupling DOE, being sensitive to the first polarization state, couples the image light impinging on the backside of the DOE into the waveguide display.

<FIG> and <FIG> show a fourth illustrative embodiment of a display engine <NUM> in which a PBS cube <NUM> is arranged with an optical stack <NUM> that comprises two PBS layers <NUM> and <NUM> (e.g., coatings) that are located on either side of an absorptive polarizer <NUM>. The absorptive polarizer is polarization sensitive (e.g., absorbs/attenuates s-polarized light). The stack <NUM> is shown in an exploded view for sake of clarity in exposition in <FIG> and <FIG>. A fast scan MEMS scanner <NUM> and quarter wave plate <NUM> are disposed at the top of PBS cube <NUM> and a slow scan MEMS scanner <NUM> and quarter wave plate <NUM> are disposed at the bottom of the PBS cube.

As shown in <FIG>, image light <NUM> in a first polarization state (e.g., s-polarized) enters the PBS cube <NUM> and is reflected by the first PBS layer <NUM> upwards to the fast scan MEMS scanner <NUM>. Image light which may leak through the first PBS layer (as indicated by arrow <NUM>) can be attenuated by the absorptive polarizer layer <NUM>. The light reflected by the first PBS layer travels upward to the fast scan MEMS scanner where it is reflected toward the slow scan MEMS scanner <NUM>. As the light makes two passes through the top quarter wave plate <NUM>, it changes to the orthogonal p-polarization state and is passed by the first and second PBS layers <NUM> and <NUM>.

Image light reflected upward by the slow scan MEMS scanner <NUM> is changed back to s-polarized by virtue of its two passes through the bottom quarter wave plate <NUM>. The second PBS layer <NUM> reflects the image light to the in-coupling DOE <NUM>. Image light which may leak through the second PBS layer (as indicated by arrow <NUM>) can be attenuated by the absorptive polarizer layer <NUM>.

The PBS cube <NUM> with the optical stack <NUM> may be useful in some implementations to reduce unscanned light leaking into the waveguide display which may cause ghosting and reduce overall image quality. Typically, the PBS coatings <NUM> and <NUM> and the absorptive polarizer have extinction ratios exceeding <NUM> to <NUM>. Accordingly, as the s-polarized image light makes multiple passes through the PBS coatings and absorptive polarizer, the intensity of the leakage into the waveguide display may be significantly reduced (i.e., by six orders of magnitude). In some cases, the absorptive polarizer can be doubled up in the PBS cube to provide additional control over light leakage.

Each embodiment of the display engine described above may be utilized in mixed-reality or virtual-reality applications. <FIG> shows one particular illustrative example of a mixed-reality or virtual-reality HMD device <NUM>, and <FIG> shows a functional block diagram of the device <NUM>. HMD device <NUM> comprises one or more lenses <NUM> that form a part of a see-through display subsystem <NUM>, so that images may be displayed using lenses <NUM> (e.g. using projection onto lenses <NUM>, one or more waveguide systems, such as a near-eye optical display system, incorporated into the lenses <NUM>, and/or in any other suitable manner). HMD device <NUM> further comprises one or more outward-facing image sensors <NUM> configured to acquire images of a background scene and/or physical environment being viewed by a user, and may include one or more microphones <NUM> configured to detect sounds, such as voice commands from a user. Outward-facing image sensors <NUM> may include one or more depth sensors and/or one or more two-dimensional image sensors. In alternative arrangements, as noted above, a mixed reality or virtual reality display system, instead of incorporating a see-through display subsystem, may display mixed reality or virtual reality images through a viewfinder mode for an outward-facing image sensor.

The HMD device <NUM> may further include a gaze detection subsystem <NUM> configured for detecting a direction of gaze of each eye of a user or a direction or location of focus, as described above. Gaze detection subsystem <NUM> may be configured to determine gaze directions of each of a user's eyes in any suitable manner. For example, in the illustrative example shown, a gaze detection subsystem <NUM> includes one or more glint sources <NUM>, such as infrared light sources, that are configured to cause a glint of light to reflect from each eyeball of a user, and one or more image sensors <NUM>, such as inward-facing sensors, that are configured to capture an image of each eyeball of the user. Changes in the glints from the user's eyeballs and/or a location of a user's pupil, as determined from image data gathered using the image sensor(s) <NUM>, may be used to determine a direction of gaze.

In addition, a location at which gaze lines projected from the user's eyes intersect the external display may be used to determine an object at which the user is gazing (e.g. a displayed virtual object and/or real background object). Gaze detection subsystem <NUM> may have any suitable number and arrangement of light sources and image sensors. In some implementations, the gaze detection subsystem <NUM> may be omitted.

The HMD device <NUM> may also include additional sensors. For example, HMD device <NUM> may comprise a global positioning system (GPS) subsystem <NUM> to allow a location of the HMD device <NUM> to be determined. This may help to identify real-world objects, such as buildings, etc. that may be located in the user's adjoining physical environment.

The HMD device <NUM> may further include one or more motion sensors <NUM> (e.g., inertial, multi-axis gyroscopic, or acceleration sensors) to detect movement and position/orientation/pose of a user's head when the user is wearing the system as part of a mixed reality or virtual reality HMD device. Motion data may be used, potentially along with eye-tracking glint data and outward-facing image data, for gaze detection, as well as for image stabilization to help correct for blur in images from the outward-facing image sensor(s) <NUM>. The use of motion data may allow changes in gaze direction to be tracked even if image data from outward-facing image sensor(s) <NUM> cannot be resolved.

In addition, motion sensors <NUM>, as well as microphone(s) <NUM> and gaze detection subsystem <NUM>, also may be employed as user input devices, such that a user may interact with the HMD device <NUM> via gestures of the eye, neck and/or head, as well as via verbal commands in some cases. It may be understood that sensors illustrated in <FIG> and <FIG> and described in the accompanying text are included for the purpose of example and are not intended to be limiting in any manner, as any other suitable sensors and/or combination of sensors may be utilized to meet the needs of a particular implementation. For example, biometric sensors (e.g., for detecting heart and respiration rates, blood pressure, brain activity, body temperature, etc.) or environmental sensors (e.g., for detecting temperature, humidity, elevation, UV (ultraviolet) light levels, etc.) may be utilized in some implementations.

The HMD device <NUM> can further include a controller <NUM> such as one or more processors having a logic subsystem <NUM> and a data storage subsystem <NUM> in communication with the sensors, gaze detection subsystem <NUM>, display subsystem <NUM>, and/or other components through a communications subsystem <NUM>. The communications subsystem <NUM> can also facilitate the display system being operated in conjunction with remotely located resources, such as processing, storage, power, data, and services. That is, in some implementations, an HMD device can be operated as part of a system that can distribute resources and capabilities among different components and subsystems.

The storage subsystem <NUM> may include instructions stored thereon that are executable by logic subsystem <NUM>, for example, to receive and interpret inputs from the sensors, to identify location and movements of a user, to identify real objects using surface reconstruction and other techniques, and dim/fade the display based on distance to objects so as to enable the objects to be seen by the user, among other tasks.

The HMD device <NUM> is configured with one or more audio transducers <NUM> (e.g., speakers, earphones, etc.) so that audio can be utilized as part of a mixed reality or virtual reality experience. A power management subsystem <NUM> may include one or more batteries <NUM> and/or protection circuit modules (PCMs) and an associated charger interface <NUM> and/or remote power interface for supplying power to components in the HMD device <NUM>.

It may be appreciated that the HMD device <NUM> is described for the purpose of example, and thus is not meant to be limiting. It may be further understood that the display device may include additional and/or alternative sensors, cameras, microphones, input devices, output devices, etc. than those shown without departing from the scope of the present arrangement. Additionally, the physical configuration of an HMD device and its various sensors and subcomponents may take a variety of different forms without departing from the scope of the present arrangement.

As shown in <FIG>, the compact display engine with MEMS scanners can be used in a mobile or portable electronic device <NUM>, such as a mobile phone, smartphone, personal digital assistant (PDA), communicator, portable Internet appliance, hand-held computer, digital video or still camera, wearable computer, computer game device, specialized bring-to-the-eye product for viewing, or other portable electronic device. As shown, the portable device <NUM> includes a housing <NUM> to house a communication module <NUM> for receiving and transmitting information from and to an external device, or a remote system or service (not shown).

The portable device <NUM> may also include an image processor <NUM> using one or more processors for handling the received and transmitted information, and a virtual display system <NUM> to support viewing of images. The virtual display system <NUM> can include a micro-display or an imager <NUM>, a display engine <NUM> and a waveguide display <NUM>. The image processor <NUM> may be operatively connected to the imager <NUM> to provide image data, such as video data so that images may be displayed using the display engine <NUM> and waveguide display <NUM>. An EPE may be included in the waveguide display <NUM>.

Various exemplary embodiments of the present compact display engine with MEMS scanners are now presented by way of illustration and not as an exhaustive list of all embodiments. An example, not according to the claimed invention, includes a near-eye optical display system configured to show images within a field of view (FOV) described by a first direction and a second direction, comprising: a waveguide display comprising one or more diffractive optical elements (DOEs) including an in-coupling DOE configured for in-coupling image light to the waveguide display; a polarizing beam splitter located proximate to the in-coupling DOE, the polarizing beam splitter configured to reflect image light having a first polarization state and transmit image light having a second polarization state; a first MEMS (micro electro mechanical system) scanner and configured to scan in the first direction of the FOV; a first quarter wave plate having an optical axis that is parallel to front and rear faces of the first quarter wave plate, the first MEMS scanner being disposed proximate to the front face and the polarizing beam splitter being disposed proximate to the rear face; a second MEMS scanner configured to scan in the second direction of the FOV; and a second quarter wave plate having an optical axis that is parallel to front and rear faces of the second quarter wave plate, the second MEMS scanner being disposed proximate to the front face and the polarizing beam splitter being disposed proximate to the rear face, wherein image light from an imager incident on the polarizing beam splitter propagates to each of the first and second MEMS scanners in succession so that the image light exits the polarizing beam splitter and is in-coupled into the waveguide display over the extent of the FOV as the first and second MEMS scanners are operated.

In another example, the FOV is rectangular and each of the first and second MEMS scanners have a single scanning axis, and the single scanning axis of the first MEMS scanner is orthogonal to the single scanning axis of the second MEMS scanner, and the first and second MEMS scanners are operated to perform raster scanning. In another example, the waveguide display further includes at least one intermediate DOE and an out-coupling DOE, wherein the one intermediate DOE provides exit pupil expansion in a first direction of the FOV and the out-coupling DOE provides exit pupil expansion in the second direction of the FOV. In another example, a propagation path of the image light in the near-eye optical display system includes two passes through each of the first and second quarter wave plates and the image light changes its state of polarization after completion of each of the two passes. In another example, the image light is polarized in the first polarization state when incident on the polarizing beam splitter. In another example, one or more of the first and second MEMS scanners are operated resonantly.

A further example, not according to the claimed invention, includes a head mounted display (HMD) device configured to display images within a field of view (FOV) having first and second directions, comprising: a polarization beam splitter (PBS) cube having twelve edges and six faces including an entrance face and an exit face, the entrance and exit faces sharing a common edge, the PBS cube configured to reflect propagating image light having a first polarization state orthogonally to a direction of propagation and further configured to transmit image light having a second polarization state parallel to the direction of propagation; an in-coupling diffractive optical element (DOE) in a waveguide display that provides exit pupil expansion in the first and second directions of the FOV, the in-coupling DOE located adjacent to the exit face of the PBS cube; a first quarter wave plate located adjacent to a face of the PBS cube that is opposite the entrance face; a second quarter wave plate located adj acent to a face of the PBS cube that is opposite the exit face; a first MEMS (micro electro mechanical system) scanner having a reflective scanning plate and configured to scan in the first direction of the FOV, the first MEMS scanner located adjacent to the first quarter wave plate; and a second MEMS scanner having a reflective scanning plate and configured to scan in the second direction of the FOV, the second MEMS scanner located adjacent to the second quarter wave plate.

In another example, not according to the claimed invention, image light entering the entrance face passes through the PBS cube, makes a first pass through the first quarter wave plate, is reflected by the first MEMS scanner, makes a second pass through the first quarter wave plate, is reflected by the PBS cube towards the face opposite the exit face, makes a first pass through the second quarter wave plate, is reflected by the second MEMS scanner towards the exit face, makes a second pass through the second quarter wave plate, is passed through the PBS cube, exits the exit face to be in-coupled by the in-coupling DOE to the waveguide display. In another example, the image light is generated by an imager comprising one or more lasers and the image light is in the second state of polarization when incident upon the entrance face of the PBS cube. In another example, the first and second MEMS scanners are operated in combination to provide raster scanning through a fast axis and a slow axis. In another example, the waveguide display includes at least one intermediate DOE configured to provide exit pupil expansion in the first direction and at least one out-coupling DOE configured to provide exit pupil expansion in the second direction and out-couple the image light from the waveguide display to an eye of an HMD device user. In another example, the image light enters the entrance face of the PBS cube in a direction that is parallel to a plane of the in-coupling DOE.

A further example includes a device configured to control image light associated with virtual images within a field of view (FOV), comprising: an imager configured to produce the image light; a waveguide display including a polarization-sensitive in-coupling diffractive optical element (DOE) configured to in-couple image light into the waveguide display or transmit image light according to a state of polarization of the image light, at least one intermediate DOE configured to expand an exit pupil of the image light in a first direction of the FOV, and an out-coupling DOE configured to expand the exit pupil of the image light in a second direction of the FOV and further configured to out-couple image light out of the waveguide display to an eye of a user of the device; a pair of MEMS (micro electro mechanical system) scanners operatively coupled to perform raster scanning of the virtual images in the FOV using the image light from the imager, the MEMS scanners further providing the image light to the in-coupling DOE in the waveguide display; and a polarizing beam splitter device configured to direct image light to either a first MEMS scanner in the pair or a second MEMS scanner in the pair according to a state of polarization of the image light.

In another example, the polarizing beam splitter device includes a polarizing beam splitting cube comprising two right angle prisms joined along a plane defined by each prism's hypotenuse, the plane including a polarization-sensitive material configured to reflect or transmit image light based on the state of polarization, in which the reflected image light and transmitted image light have an orthogonal separation and the pair of MEMS scanners are respectively located with respect to the cube to receive the separated beams of image light. In another example, the device further includes a pair of quarter wave plates, each quarter wave plate respectively disposed adjacent to a MEMS scanner in the pair of MEMS scanners so that image light makes two passes through each quarter wave plate when the image light is reflected by each MEMS scanner, the state of polarization of the image light changing to an orthogonal state after each of the two passes. In another example, the polarization-sensitive in-coupling DOE includes at least a portion that is configured as one of a Bragg grating or grating having periodic features in two directions. In another example,.

The compact display engine with MEMS scanners may also be utilized in non-portable devices, such as gaming devices, multimedia consoles, personal computers, vending machines, smart appliances, Internet-connected devices, and home appliances, such as an oven, microwave oven and other appliances, and other non-portable devices.

Claim 1:
A device (<NUM>) configured to control image light associated with virtual images within a field of view (FOV), comprising:
an imager (<NUM>) configured to produce the image light;
a waveguide display (<NUM>) including a polarization-sensitive in-coupling diffractive optical element (DOE) (<NUM>) configured to in-couple image light into the waveguide display (<NUM>) or transmit image light according to a state of polarization of the image light, at least one intermediate DOE (<NUM>) configured to expand an exit pupil of the image light in a first direction of the FOV, and an out-coupling DOE (<NUM>) configured to expand the exit pupil of the image light in a second direction of the FOV and further configured to out-couple image light out of the waveguide display (<NUM>) to an eye of a user of the device (<NUM>); characterised in that the device further comprises
a pair of MEMS (micro electro mechanical system) scanners (<NUM>, <NUM>) operatively coupled to perform raster scanning of the virtual images in the FOV using the image light from the imager (<NUM>), the MEMS scanners (<NUM>, <NUM>) further providing the image light to the in-coupling DOE (<NUM>) in the waveguide display (<NUM>); and
a polarizing beam splitter device (<NUM>) configured to direct image light to a first MEMS scanner (<NUM>) in the pair when a state of polarization of the image light is a first state of polarization and to a second MEMS scanner (<NUM>) in the pair when the state of polarization of the image light is a second state of polarization,
a pair of quarter wave plates, each quarter wave plate respectively disposed adjacent to a MEMS scanner in the pair of MEMS scanners so that image light makes two passes through each quarter wave plate when the image light is reflected by each MEMS scanner, the state of polarization of the image light changing to an orthogonal state after each of the two passes,
wherein the polarization-sensitive in-coupling DOE is located between the polarizing beam splitter device and one of the MEMS scanners in the pair of MEMS scanners.