Patent Description:
The present invention relates to eye tracking.

Optical arrangements for near eye display (NED), head mounted display (HMD) and head up display (HUD) require large aperture to cover the area where the observer's eye is located (commonly referred to as the eye motion box - or EMB). In order to implement a compact device, the image that is to be projected into the observer's eye is generated by a small optical image generator (projector) having a small aperture that is multiplied to generate a large aperture.

An approach to aperture multiplication in one dimension has been developed based on a parallel-faced slab of transparent material within which the image propagates by internal reflection. Part of the image wavefront is coupled out of the slab, either by use of obliquely angled partial reflectors or by use of a diffractive optical element on one surface of the slab. Such a slab is referred herein as a light-guide optical element (LOE), light transmitting substrate, or waveguide. The principles of such aperture multiplication are illustrated schematically in <FIG>, which shows a light-guide optical element <NUM> having a pair of parallel faces <NUM>, 26A for guiding light by internal reflection. A projected image <NUM>, as represented here schematically by a beam of illumination <NUM> including sample rays 18A and 18B which span the beam, is coupled into the light-guide optical element <NUM>, as illustrated here schematically by a first reflecting surface <NUM>, so as to generate reflected rays <NUM> which are trapped by internal reflection within the substrate, generating also rays <NUM>. The image propagates along the substrate by repeated internal reflection, impinging on a sequence of partially reflecting surfaces <NUM> at an oblique angle (αsur) to the parallel faces <NUM>, <NUM>A, where part of the image intensity is reflected so as to be coupled out of the substrate as rays 32A, 32B toward the eye <NUM> of an observer. In order to minimize unwanted reflections which might give rise to ghost images, the partially reflecting surfaces <NUM> are preferably coated so as to have low reflectance for a first range of incident angles, while having the desired partial reflectivity for a second range of incident angles, where a ray with a small inclination to the normal to a partially reflective surface <NUM> (represented here as angle βref) is split in order to generate a reflected ray for coupling out, while a high inclination (to the normal) ray is transmitted with negligible reflection.

The projected image <NUM> is a collimated image, i.e., where each pixel is represented by a beam of parallel rays at a corresponding angle, equivalent to light from a scene far from the observer (the collimated image is referred to as being "collimated to infinity"). The image is represented here simplistically by rays corresponding to a single point in the image, typically a centroid of the image, but in fact includes a range of angles to each side of this central beam, which are coupled in to the substrate with a corresponding range of angles, and similarly coupled out at corresponding angles, thereby creating a field of view corresponding to parts of the image arriving in different directions to the eye <NUM> of the observer.

An optical function which could be useful for NED, HMD or HUD designs is eye tracking, or sensing the direction the eye of the observer is looking relative to the direction of the head (commonly referred to as the gaze direction). Past eye tracking approaches relied on imaging the EMB via one or more off-axis cameras looking from the side toward the EMB. In order to reduce user discomfort, the cameras should be of relatively small size, which can limit the EMB imaging performance. The small camera size, together with the general difficulty of deriving the gaze direction from EMB images sampled at high off-axis angles, results in relatively low performance of such eye tracking approaches.

Document <CIT> Al discloses a head-mounted display device for providing augmented reality contents to a wearer includes an eye tracker, a light projector, a beam steerer and a combiner. The eye tracker is configured to determine a position of a pupil of an eye of the wearer. The light projector is configured to project light for rendering images. The beam steerer is configured to change a direction of the light from the light projector based on the position of the pupil. The combiner is configured to combine the light from the light projector and light from an outside of the head-mounted display device for providing an overlap of the rendered image and a real image that corresponds to the light from the outside of the head-mounted display device.

Aspects of the present invention provide an eye tracker and corresponding method for tracking the gaze direction of a human eye based on imaging the eye via a light-guide optical element, and are particularly suitable for integrating as part of a NED, HMD or HUD.

According to the teachings of an embodiment of the present invention, there is provided an apparatus that comprises: a light-transmitting substrate having at least two parallel major surfaces for guiding light by internal reflection, a first of the major surfaces being deployed in facing relation to an eye; an optical element associated with the first of the major surfaces, the optical element configured for applying optical power to incident light in accordance with at least one property of the incident light, wherein the at least one property is the wavelength, such that the optical element applies non-zero optical power to incident light of a first type returned from the eye within a first not visible optical spectrum so as to collimate the incident light of the first type and such that the optical element applies substantially zero optical power to incident light of a second type within a second visible optical spectrum; an optical coupling configuration associated with the substrate and configured for: coupling-in a proportion of light of the first type, collimated by the optical element and incident on the first of the major surfaces, so as to propagate within the substrate by total internal reflection, and coupling-out a proportion of light of the second type propagating within the substrate; optics associated with the substrate and configured for converting the collimated light of the first type into converging beams of captured light; an optical sensor deployed for sensing the captured light; and at least one processor electrically associated with the optical sensor and configured to process signals from the optical sensor to derive a current gaze direction of the eye, wherein the optical element is positioned between the light-transmitting substrate and the eye.

Optionally, the at least one property of the incident light includes a polarization direction of the incident light.

Optionally, the at least one property of the incident light includes a polarization direction of the incident light and a region of the electromagnetic spectrum occupied by the incident light.

Optionally, the light of the first type includes components of light that are polarized in a first polarization direction, and wherein the light of the second type is polarized in a second polarization direction.

Optionally, the light of the first type includes components of light that are polarized in a first polarization direction and is within a first optical spectrum, and wherein the light of the second type is polarized in a second polarization direction and is within a second optical spectrum.

Optionally, the apparatus further comprises: a polarizer associated with a second of the major surfaces of the substrate.

Optionally, the substrate is deployed with the first of the major surfaces at an eye relief distance from the eye, and wherein the optical element has a focal length approximately equal to the eye relief distance.

Optionally, the apparatus further comprises: a second optical coupling configuration associated with the optics and configured for: coupling-out a proportion of light of the first type propagating within the substrate such that the coupled-out light is received by the optics, and coupling-in a proportion of light of the second type, from a display source, so as to propagate within the substrate by internal reflection.

Optionally, the apparatus further comprises: an illumination arrangement deployed to illuminate the eye with light of the first type.

Optionally, the apparatus further comprises: an image projector coupled to the substrate so as to introduce collimated light of the second type corresponding to an image into the substrate such that the coupled-in collimated light of the second type propagates by internal reflection within the substrate and is coupled out of the substrate toward the eye by the optical coupling configuration.

Optionally, the image projector includes a reflective-display device that produces polarized light in response to illumination from a polarized source of light, and wherein the polarized light produced by the reflective-display device is collimated by the optics.

Optionally, the optical coupling configuration includes a plurality of partially reflective surfaces deployed within the substrate obliquely to the major surfaces of the substrate.

Optionally, light of the first type propagates within the substrate in a first propagation direction, and wherein light of the second type propagates within the substrate in a second propagation direction opposite the first propagation direction.

It is not claimed an apparatus that comprises: a light-transmitting substrate having a pair of parallel major surfaces for guiding light by internal reflection, a first of the major surfaces being deployed in facing relation to an eye of a viewer; a lens associated with the first of the major surfaces, the lens configured for: applying optical power to incident light of a first type so as to collimate the incident light of the first type, wherein the incident light of the first type is within a first optical spectrum and includes components of light that has polarization in a first polarization direction, and applying substantially no optical power to incident light of a second type, wherein the incident light of the second type is within a second optical spectrum and has polarization in a second polarization direction; an illumination arrangement deployed to illuminate the eye with light of the first type such that a proportion of the light of the first type is reflected by the eye back toward the lens so as to be collimated by the lens; an optical module including: a reflective-display device that produces light of the second type, corresponding to an image, in response to illumination from a source of light, optics configured for collimating the light produced by the reflective-display device so as to produce collimated light of the second type, and an optical sensor; an optical coupling configuration configured for coupling the collimated light of the second type into the substrate so as to propagate within the substrate by internal reflection in a first propagation direction; a plurality of partially reflective surface deployed within the substrate obliquely to the major surfaces of the substrate, the partially reflective surfaces configured for: coupling-out a proportion of light of the second type, propagating within the substrate in the first propagation direction, and coupling-in a proportion of the collimated light of the first type incident on the first of the major surfaces, so as to propagate within the substrate in a second propagation direction, wherein the optical coupling configuration is further configured for coupling-out the propagating light of the first type; and at least one processor electrically coupled to the optical sensor, wherein the optics of the optical module are further configured for receiving the light coupled-out by the optical coupling configuration and for converting the coupled-out light into converging beams of captured light, and wherein the optical sensor is configured for sensing the captured light, and wherein the at least one processor is configured to process signals from the optical sensor to derive a current gaze direction of the eye.

It is not claimed an apparatus that comprises: a first light-transmitting substrate having at least two substantially parallel major surfaces for guiding light by internal reflection, a first of the major surfaces being deployed in facing relation to an eye of a viewer; an at least partially reflective surface deployed within the first substrate obliquely to the major surfaces, the at least partially reflective surface configured to couple incident light rays that are incident on the first of the major surfaces within a coupling-in region so as to propagate within the first substrate by internal reflection, wherein the incident light rays are in a first optical spectrum and emanate from the eye in response to illumination of the eye, and wherein the incident light rays include at least a first set of light rays and a second set of light rays, the first set of light rays having an angular distribution spanning at least a portion of the coupling-in region in a first dimension, and the second set of light rays spanning at least a portion of the coupling-in region in a second dimension; a coupling-out arrangement configured for coupling-out the light rays propagating within the first substrate; an optical module including: at least one lens having a first focal length in a first dimension of the lens and a second focal length in a second dimension of the lens, and configured for: converting the coupled-out light rays corresponding to the first set of light rays into non-converging beams of captured light having an angular distribution indicative of the angular distribution of the first set of light rays, and converting the coupled-out light rays corresponding to the second set of light rays into converging beams of captured light, and an optical sensor positioned at a distance from the lens substantially equal to the first focal length and configured for sensing the captured light; and at least one processor electrically coupled to the optical sensor configured to process signals from the optical sensor to derive a current gaze direction of the eye.

It is not claimed that the apparatus further comprises: an illumination arrangement deployed to illuminate the eye with light in the first optical spectrum
It is not claimed that the apparatus further comprises: a second light-transmitting substrate having a plurality of surfaces including mutually parallel first and second major surfaces for guiding light by internal reflection, the first major surface of the second substrate being deployed in facing relation to the eye, and the second major surface of the second substrate being deployed in facing relation to the first of the major surfaces of the first substrate; and a coupling-out configuration associated with the second substrate, the coupling-out configuration configured to couple a proportion of light in a second optical spectrum, different from the first optical spectrum and propagating within the second substrate, out of the second substrate toward the eye.

It is not claimed that the apparatus further comprises: an image projector coupled to the second substrate and configured to generate collimated light in the second optical spectrum corresponding to an image such that the collimated light propagates by internal reflection within the second substrate and is coupled out of the second substrate toward the eye by the coupling-out configuration.

It is not claimed that the apparatus further comprises: a coupling-in arrangement associated with the image projector and the second substrate configured to couple the collimated light generated by the image projector into the second substrate.

It is not claimed that the coupling-out configuration includes a plurality of partially reflective surfaces deployed within the second substrate obliquely to the major surfaces of the second substrate.

It is not claimed that the coupling-out configuration includes a diffractive optical element associated with one of the major surfaces of the second substrate.

Unless otherwise defined herein, all technical and/or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention pertains. Although methods and materials similar or equivalent to those described herein may be used in the practice or testing of embodiments of the invention, exemplary methods and/or materials are described below.

Some embodiments of the present invention are herein described, by way of example only, with reference to the accompanying drawings. With specific reference to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of embodiments of the invention.

Attention is now directed to the drawings, where like reference numerals or characters indicate corresponding or like components. In the drawings:.

Embodiments of the present invention provide various apparatus and corresponding methods for tracking the gaze direction of a human eye based on imaging the eye and/or identifying an angular distribution of light reflected by the eye via a light-guide optical element.

The principles and operation of the various eye tracking apparatus according to present invention may be better understood with reference to the drawings accompanying the description.

By way of introduction, in many applications, particularly in the context of head-up or near-eye displays, it is useful to provide an eye tracking arrangement for determining the gaze direction of the user. One common approach for performing eye tracking is to sample an image of the eye, typically for the purpose of determining the pupil position within the image, and thereby deriving the orientation of the eye. It would be particularly advantageous to employ a light-guide optical element operating on principles similar to those of <FIG> to sample images for eye tracking.

Eye tracking solutions employing a light-guide optical element operating on such principles or similar such principles are described herein. In one set of solutions according to certain aspects of the present invention, the eye is imaged by way of coupling light, reflected from the eye (referred to as light of a first type), back into the light-guide optical element, whereby the light propagates along a reverse path through the light-guide optical element, in a reverse propagation direction of image light from an image projector (referred to as light of a second type), and is focused onto an optical sensor deployed in the image projector, where signals produced by the optical sensor, in response to sensing the light, are processed by a processing system to derive the gaze direction. Since the eye is not located at infinity from the light-guide optical element (but rather at an eye relief distance, typically on the order of approximately <NUM> millimeters), the light reflected from the eye is collimated by an optical element, preferably a polarization and/or spectrally selective lens that discriminates between the light of the first and second types, prior to being coupled into the light-guide optical element in order to accurately derive the gaze direction from the light focused on the optical sensor.

In another set of solutions according to aspects of the present invention, the gaze direction is determined by way of a specialized partially-reflective surface, preferably in a dedicated light-guide optical element separate from the LOE through which the projected image propagates, which couples uncollimated light, reflected from the eye, into the light-guide optical element, whereby the coupled-in in light propagates along a reverse path through the light-guide optical element and is coupled out to an optical module that includes a lens having two focal lengths in respective orthogonal dimensions which directs the coupled-out light to an optical sensor.

Referring now to the drawings, <FIG> illustrate various aspects of the structure and operation of an apparatus, generally designated <NUM>, constructed and operative according to various embodiments of the present invention, for displaying an image and for deriving a gaze direction of a human eye <NUM> by way of a collimating optical element <NUM> (referred to herein after as lens <NUM>) deployed between the eye <NUM> and a light-guide optical element (LOE) <NUM>. The LOE <NUM> is formed from transparent material and has a pair of parallel faces (planar major surfaces) <NUM>, <NUM> for guiding light by internal reflection (preferably total internal reflection). The LOE <NUM> is deployed with one of the parallel faces <NUM> in facing relation to the eye <NUM>, where the eye <NUM> is located in the EMB <NUM> at an eye relief (ER) distance <NUM> from the face <NUM>. An optical coupling configuration, implemented as a set of partially reflective surfaces <NUM>, is associated with the LOE <NUM> and is configured for coupling-in a proportion of light incident on the face <NUM> within a coupling-in region so as to propagate within the LOE <NUM> by (total) internal reflection. In particular, the partially reflective surfaces <NUM> are deployed within the LOE <NUM> (i.e., between the faces <NUM>, <NUM>) obliquely to the parallel faces <NUM>, <NUM>. The coupling-in region of the LOE <NUM>, also referred to as the "active region" or "active area", is generally defined as the region spanned by the projection of the partially reflective surfaces <NUM> in the plane of the face <NUM>.

The lens <NUM> is associated with the face <NUM> (by way of optical attachment to the LOE <NUM>) such that the lens <NUM> is positioned between the LOE <NUM> and the eye <NUM>. The lens <NUM> preferably has a focal length approximately equal to the ER <NUM>. Light reflected from the eye <NUM> (in response to illumination of the eye <NUM> by an illumination arrangement <NUM>) is collimated by the lens <NUM> whereupon the collimated light is incident on the face <NUM> and is coupled into to the LOE <NUM> by the partially reflective surfaces <NUM> so as to propagate within the LOE <NUM> by internal reflection. An optical element <NUM> (referred to hereinafter as lens <NUM>) is associated with the LOE <NUM> so as to receive the captured light propagating within the LOE <NUM> and to convert collimated light (sets of parallel light rays) propagating within the LOE <NUM> into converging beams of captured light. Preferably, the lens <NUM> is integrated into an optical module <NUM> together with an optical sensor <NUM> which is configured for sensing the captured light, and the lens <NUM> is associated with the LOE <NUM> via an optical coupling configuration <NUM> that couples the captured light propagating within the LOE <NUM> out of the LOE <NUM> to the optical module <NUM>. A processing system <NUM>, that includes at least one computerized processor <NUM> coupled to a storage medium <NUM> (such as a computer memory or the like), is electrically associated with the optical sensor <NUM>, and is configured to process signals from the optical sensor <NUM> to derive a current gaze direction of the eye <NUM>.

The optical coupling configuration <NUM> may be any coupling arrangement which deflects incident light out of the LOE <NUM> and into the optical module <NUM>. Suitable optical coupling configurations include, but are not limited to, a reflecting surface (as shown schematically in <FIG>) and a prism (as shown schematically in <FIG>).

Generally speaking, the eye <NUM> is illuminated with light by the illumination arrangement <NUM>. As will be discussed, the illumination arrangement <NUM> is configured to illuminate the eye <NUM> with light having wavelengths outside of the photopic region of the electromagnetic spectrum. In other words, the illumination arrangement <NUM> is configured to illuminate the eye <NUM> with light that is not visible to the human eye. Reflection from the human eye, and in particular reflection from the retina of the eye, is substantially higher in the near infrared than at visible wavelengths. Accordingly, it is preferable that the illumination arrangement <NUM> is configured to illuminate the eye <NUM> with light having wavelengths in the near infrared (NIR) region of the electromagnetic spectrum. In addition, and as will be discussed in detail in subsequent sections of the present disclosure, the illumination arrangement <NUM> is also preferably configured to illuminate the eye <NUM> such that the light reflected by eye <NUM> in response to illumination from the illumination arrangement <NUM> includes at least components of light having a particular polarization direction (typically p-polarized) relative to the surface of the lens <NUM>.

Referring now specifically to <FIG> and <FIG>, this shows the traversal of light rays from the eye <NUM> to the optical sensor <NUM> via the LOE <NUM>. In general, light propagating within the LOE <NUM> from the eye <NUM> to the optical sensor <NUM> is referred to as propagating within the LOE <NUM> in a reverse propagation direction (referred to interchangeably as a first/second propagation direction, first/second direction, or reverse direction), whereas image light propagating within the LOE <NUM> from the image projector to the eye <NUM> is referred to as propagating within the LOE <NUM> in a forward propagation direction (referred to interchangeably as a second/first propagation direction, second/first direction, or forward direction) opposite the reverse propagation direction. A proportion of the intensity of the light from the illumination arrangement <NUM> incident on the eye <NUM> is reflected by the eye <NUM>. The reflected light emanating from the eye <NUM> is schematically represented in <FIG> and <FIG> as sample light rays 114A - 114F. Light emanating from the eye <NUM> is collimated by the lens <NUM>, where the collimated light is schematically represented as light rays 116A - 116F (each of the respective light rays 114A - 114F has a corresponding collimated light ray 116A - 116F). The collimated light rays 116A - <NUM>F are incident on the face <NUM> of the LOE <NUM> generally normal to the face <NUM>, and are coupled into the LOE <NUM> by the partially reflective surfaces <NUM> so as to generate reflected rays <NUM> (down-going rays) which are trapped by internal reflection within the LOE <NUM>, generating also (up-going) rays <NUM>. The light reflected from the eye <NUM> propagates along the substrate until it reaches the optical coupling configuration <NUM> (shown schematically as a reflecting surface in <FIG> and as a prism in <FIG>), which couples the light (light rays <NUM> and <NUM>) out of the LOE <NUM> as light rays 122A, 122B, and 122C to the optical module <NUM>. The lens <NUM> converts the collimated coupled-out light (rays 122A, 122B, and 122C) into converging beams of captured light so as to focus the coupled-out light (rays 122A, 122B, and 122C) onto the optical sensor <NUM>.

The optical module <NUM>, in addition to having the lens <NUM> and the optical sensor <NUM> integrated therein, preferably also includes components for generating and projecting the image into the LOE <NUM> for viewing by the eye <NUM> (similar to the projected image <NUM> in <FIG>), such that the optical module <NUM> performs the dual functionality of image projection and light focusing and sensing. As will be discussed, the lens <NUM> also functions to collimate light rays produced by a display device of the optical module <NUM>.

Referring now to <FIG>, this shows the propagation of light within the LOE <NUM> in the forward direction. Similar to as in <FIG>, a projected image <NUM>, as represented here schematically by a beam of illumination <NUM> including sample rays 142A, 142B, and 142C which span the beam, is generated by the optical module <NUM> and is coupled into the LOE <NUM> via the optical coupling configuration <NUM> (as illustrated here schematically by a reflecting surface) so as to generate reflected rays <NUM> (up-going rays) which are trapped by internal reflection within the LOE <NUM>, generating also rays <NUM> (down-going rays). The image <NUM> propagates along the LOE <NUM> by repeated internal reflection between the faces <NUM>, <NUM>, impinging the partially reflecting surfaces <NUM> where part of the image intensity is reflected so as to be coupled out of the LOE <NUM> as rays 148A, 148B, and 148C toward the eye <NUM>. However, prior to reaching the eye <NUM>, the light rays 148A - 148C necessarily pass through the lens <NUM>.

While it is critical for the lens <NUM> to apply optical power to light emanating from the eye <NUM> so as to collimate the light rays 114A - 114F in order to enable accurate sensing of the captured light (by the optical sensor <NUM>) and processing (by the processing system <NUM>) of the signals from the optical sensor <NUM> to derive a current gaze direction of the eye <NUM>, it is equally critical that the lens <NUM> applies no optical power to the image light propagating from the optical module <NUM> to the eye <NUM> via the LOE <NUM> as the application of optical power to the light rays 148A, 148B, and 148C would distort the projected image <NUM> when viewed by the eye <NUM>. Therefore, it is a particular feature of the present embodiments to design the lens <NUM> such that the lens can discriminate between two types of light (light reflected from the eye, represented by light rays 114A - 114F, that propagates via the LOE <NUM> to the focusing and sensing components of the optical module <NUM>, referred to as light of a first type, and image light from the image projection components of the optical module <NUM>, represented by light rays 142A - 142C, referred to as light of a second type), and apply optical power to only one of those types of light (namely the light of the first type, i.e., the reflected eye light). Within the context of this document, the terms "light of the first type", "light waves of the first type", "first type of light", "first type of light waves", and variations thereof are used interchangeably. Also, within the context of this document, the terms "light of the second type", "light waves of the second type", "second type of light", "second type of light waves", and variations thereof, are used interchangeably.

According to certain preferred embodiments, the discrimination is performed based on at least one property of the light that is incident on the lens <NUM>. In other words, the lens <NUM> is designed such that the lens <NUM> selectively applies optical power to incident light in accordance with at least one property (feature) of the incident light. In certain embodiments, one property - for example the wavelength (i.e., the optical spectrum) of the incident light - is used as a basis to discriminate between the first and second types of light, while in other embodiments another property - for example the polarization direction or polarization direction of components of the incident light - is used as a basis to discriminate between the first and second types of light, while yet in other preferred embodiments both the optical spectrum (wavelength) and the polarization direction of the incident light is used as a basis to discriminate between the first and second types of light.

It is generally noted that in contrast to the light that illuminates the eye <NUM>, the image light <NUM> (light of the second type) has wavelengths in the photopic region of the electromagnetic spectrum (i.e., between <NUM> nanometers (nm) and approximately <NUM>). Therefore, the lens <NUM> can be designed in a way such that optical power is only applied to light having wavelengths outside of the photopic region of the electromagnetic spectrum. In addition, in many applications it is preferable that the image light projected by the optical module <NUM> is linearly polarized in a specific polarization direction (preferably s-polarized). As such, the lens <NUM> may be designed such that the lens <NUM> applies optical power to polarized light having a polarization direction rotated with respect to the polarization direction of the coupled-out image light projected by the optical module <NUM>. Accordingly, the lens <NUM> is preferably designed to be polarization and spectrally selective such that optical power is applied to incident light waves of the first type so as to collimate the incident light waves of the first type, and such that the lens <NUM> does not apply optical power to incident light waves of the second type, and in which the incident light waves of the first type have components in a first polarization direction (e.g., p-polarized) and have wavelength in a first optical spectrum (e.g., the NIR region of the electromagnetic spectrum), and in which the incident light waves of the second type have a second polarization direction rotated relative to the first polarization direction (e.g., s-polarized) and have wavelength in a second optical spectrum (e.g., the photopic (or visible light) region of the electromagnetic spectrum). To this end, for the first type of incident light waves, the lens <NUM> has a focal length approximately equal to the ER <NUM>.

In the aforementioned example configuration of the lens <NUM>, the light rays 114A - 114F (the first type of light) represent the p-polarized (relative to the surface of the lens <NUM>) components of the light emanating from the eye <NUM> and have wavelengths in the NIR region of the electromagnetic spectrum, whereas the light rays 148A - 148C (the light of the second type) that are coupled-out from the LOE <NUM> are s-polarized (relative to the surface of the lens <NUM>) and have wavelengths in the visible region of the electromagnetic spectrum. As a result of the polarization and wavelength dependent optical power discrimination performed by the lens <NUM>, the lens <NUM> applies optical power to p-polarized NIR light waves so as to collimate the light rays 116A - 116F (the first type of light), and does not apply optical power to the s-polarized visible image light waves coupled-out of the LOE <NUM> such that the light rays 148A - 148C (the second type of light) coupled out of the LOE <NUM> (by the partially reflective surfaces <NUM>) pass through the lens <NUM> without being distorted by the lens <NUM>. Furthermore, the lens <NUM> does not apply optical power to any s-polarized components of the NIR light reflected from the eye <NUM>.

One particular class of materials that exhibit birefringent (polarization) and/or spectral properties are liquid crystals, which have different effects on light of different polarizations and in certain instances difference wavelengths. For example, nematic phase liquid crystal molecules react differently to incident light of two different linear polarizations (s-polarization and p-polarization). In an exemplary but non-limiting implementation, the lens <NUM> is implemented as a nematic phase liquid crystal lens composed of layers of liquid crystal material. The layers of liquid crystal material assume a state which provides a tunable focal length whereby the lens <NUM> has a prescribed focal length for polarized light in one polarization direction (e.g., p-polarized) so as to act as a collimator for that light, and the lens <NUM> applies no optical power to light of the orthogonal polarization (e.g., s-polarized). Each liquid crystal molecule in nematic phase liquid crystals has a different susceptibility to each linear polarization, and hence a different refractive index of the liquid crystal molecule can be induced. As such, incident light in one polarization direction "sees" no change in refractive index, whereas the incident light in the other polarization will "see" a change in refractive index thereby inducing a lens effect for light of that polarization.

In twisted nematic liquid crystals, each liquid crystal molecule has a different susceptibility to each circular polarization (e.g., right-hand circular polarization (or RHP), and left-hand circular polarization (or LHP)). Typically, the susceptibility for twisted nematic liquid crystals is such that for RHP a positive power lensing effect is induced, while for LHP a negative power lensing effect is induced. Introducing another isotropic lens with the same focal length induced by the liquid crystal lens <NUM> can double the optical power for one polarization and yield no optical power for the other polarization. It is noted since optical power is applied differently to RHP and LHP light, a quarter wave plate <NUM> is preferably deployed between the eye <NUM> and the lens <NUM> to properly rotate the circular polarization direction of the reflected light from the eye <NUM>.

Lenses constructed from liquid crystal materials are generally composed from thin diffractive-grating-type structures (similar to as in Fresnel lenses) which create diffractive dispersion of incident light. Each grating can be designed to have a larger intensity for a specific order of diffraction of that grating. The high intensity for that specific order of diffraction is chromatic (i.e., wavelength dependent). Therefore, the gratings can be designed such that for wavelengths in the NIR region, the relative intensity of <NUM>st or higher nodes of diffraction are higher than the intensity of the <NUM>th node of diffraction. In the photopic region, the high order nodes should have small intensity or no intensity at all. The grating orientation is spatially varied such that the above conditions for light in the NIR and photopic regions are satisfied, thereby creating a lensing effect, such that the lens <NUM> effectively collimates the light, and incident light having wavelength in a second optical spectrum (e.g., the photopic (or visible light) region of the electromagnetic spectrum) is essentially unaffected by the lens <NUM>. It is noted that here the grating orientation of the liquid crystal molecules are changed so as to spatially change the refractive index of the liquid crystal molecule without exploiting the birefringent properties of the liquid crystal material.

In general, the lens <NUM> may be designed to discriminate based on a combination of wavelength and polarization. However, if the spectral separation between the first and second optical spectra is large enough without adversely affecting the light from the image projector, the discrimination between the first and second types of light based solely on wavelength could be sufficient. Generally speaking, the effect of the lens <NUM> on light from image projector can be evaluated based on one or more image quality metrics, including, for example, MTF, haze, checkerboard contrast, and the like.

It is noted that the apparatus <NUM> of the present disclosure are particularly applicable when used in augmented reality (AR) systems, where the image projected by the optical module <NUM> is overlaid on the real-world scene viewable to the observer through the faces <NUM>, <NUM> and the partially reflective surfaces <NUM>. Accordingly, it is also preferable that the light waves from the real-world scene that pass through the faces <NUM>, <NUM> of the LOE <NUM> are not distorted by the lens <NUM> before reaching the eye <NUM>. To prevent the light waves from the real-world scene from being distorted by the lens <NUM>, a polarizer <NUM> that transmits only the components of incident light in the second polarization direction (e.g., s-polarized) is associated with the face <NUM>. The polarizer <NUM> and the LOE <NUM> preferably have a common direction of elongation (illustrated arbitrarily herein as corresponding to the x-axis). Preferably, the polarizer <NUM> is deployed so as to extend across the entirety (or close to the entirety) of the face <NUM> such that the light from the entire real-world field of view (corresponding to a wide angular distribution of incoming light rays) is properly polarized by the polarizer <NUM> before impinging on the face <NUM>.

The effect of the polarizer <NUM> on the real-world scene is illustrated schematically in <FIG>. As illustrated, a real-world scene image <NUM>, as represented here schematically by a beam of illumination <NUM> including sample rays 152A and 152B which span the beam, impinge on the polarizer <NUM>, which transmits only the s-polarized light components of the light rays 152A and 152B. Since the light rays 152A and 152B are s-polarized and have wavelength in the photopic region, the s-polarized light rays 152A and 152B, similar to as with the light rays 148A - 148C, pass through the lens <NUM> and reach the eye <NUM> without being distorted by the lens <NUM> (i.e., the lens <NUM> does not apply any optical power to the light rays 152A and 152B).

As discussed in the background section, in order to minimize unwanted reflections which might give rise to ghost images, the partially reflective surfaces are preferably coated so as to have low reflectance for a first range of incident angles, while having the desired partial reflectivity for a second range of incident angles. In the prior art configuration of <FIG>, these coatings are typically specific to the wavelength range and polarization of the projected image. For example, if the projected image is composed of s-polarized light having wavelength in the photopic region of the electromagnetic spectrum, the partially reflective surfaces are coated so as to have low reflectance for s-polarized light in the photopic region at a first range of incident angles, while having the desired partial reflectivity for s-polarized light in the photopic region at a second range of incident angles. This coating scheme is ideal for the configuration of <FIG> since light only propagates in the forward direction and the partially reflective surfaces <NUM> are only used to couple light out of the LOE <NUM>. However, in the configuration of <FIG>, in which a first type of light (p-polarized components of NIR light reflected from the eye <NUM>) propagates in the reverse direction and a second type of light (s-polarized photopic light from the image projector) propagates in the forward direction and the partially reflective surfaces <NUM> are configured to couple the first type of light into the LOE <NUM> and couple the second type of light out of the LOE <NUM>, a modified coating scheme should be followed to ensure the proper desired reflectivity for the first type of light. Specifically, the partially reflective surfaces <NUM> are preferably coated as in the configuration of <FIG> and additionally coated such that they have the desired reflectivity for p-polarized light in the NIR region at a prescribed range of incident angles.

As previously discussed, the optical module <NUM> performs a dual role of image projection and light focusing and sensing. The following paragraphs describe the structure and operation of the optical module <NUM> in its role as both an image projector for projecting the image <NUM>, as well as a focusing and sensing arrangement for focusing the light reflected from the eye <NUM> onto the optical sensor <NUM>.

Referring first to <FIG>, the optical module <NUM> (also referred to as the image projector <NUM>) includes an illumination prism <NUM> and a collimation-focusing prism <NUM>, each formed from a light-wave transmitting material. The illumination prism <NUM> has a number of external surfaces including a light-wave entrance surface <NUM>, an image display surface <NUM>, a light-wave exit-and-entrance surface <NUM>, and a light-wave exit surface <NUM>. A polarization selective beamsplitter configuration <NUM> is deployed within the prism <NUM> on a plane oblique to the light-wave entrance surface <NUM>. The prism <NUM> is based on two constituent prisms, namely a first constituent prism <NUM> and a second constituent prism <NUM>, where at least one of the prisms <NUM>, <NUM> is provided on the hypotenuse side with a polarizing beamsplitter (for example a wire grid beamsplitter) forming at least part of the polarization selective beamsplitter configuration <NUM>, which reflects s-polarized light and transmits p-polarized light (incident to the surface of the beamsplitter). The two hypotenuse sides of the prisms <NUM>, <NUM> are cemented to each other, to form a cemented unitary illumination prism assembly. This single cemented prism is used for illuminating a reflective-display device (for image projection) and also for directing the incoming light reflected from the eye <NUM> onto the optical sensor <NUM>, which is associated with the light-wave exit surface <NUM>. The polarizing beamsplitter can be provided via a polarization selective coating directly on one of the hypotenuse sides, or via a thin piece of material such as, for example, a sheet, foil, or glass plate, having a polarization selective coating deposited thereon, whereby the thin piece of material is attached to one of the hypotenuse sides.

In certain preferred implementations, the surfaces <NUM> and <NUM> are mutually parallel, and the surfaces <NUM> and <NUM> are mutually parallel. In certain particularly preferred implementations, the prism <NUM> is a cuboid prism, i.e., with rectangular faces orthogonal to each other, and in certain particularly preferred examples illustrated here, it is a square cuboid prism, where each constituent prism <NUM> and <NUM> has a <NUM>-degree right-angled cross-sectional shape.

The collimation-focusing prism <NUM> also has a number of external surfaces including a first light-wave exit-and-entrance surface <NUM> (aligned with and parallel to the light-wave exit-and-entrance surface <NUM>), a second light-wave exit-and-entrance surface <NUM>, a collimation-focusing surface <NUM>, and a fourth surface <NUM>. A polarization-and-spectrally selective beamsplitter configuration <NUM> is deployed within the prism <NUM> on a plane oblique to the surface <NUM>. As can be seen in <FIG>, the beamsplitter configurations <NUM> and <NUM> are in parallel planes. The prism <NUM> is based on two constituent prisms, namely a first constituent prism <NUM> and a second constituent prism <NUM>, where at least one of the prisms <NUM>, <NUM> is provided on the hypotenuse side with a polarization-and-spectrally selective beamsplitter forming at least part of the polarization-and-spectrally selective beamsplitter configuration <NUM>, which reflects p-polarized light and transmits s-polarized light having wavelengths in the first optical spectrum (e.g., NIR region) and reflects s-polarized light and transmits p-polarized light having wavelengths in the second optical spectrum (e.g., photopic (or visible light) region). The two hypotenuse sides of the prisms <NUM>, <NUM> are cemented to each other, to form a cemented unitary collimation-focusing prism assembly. This single cemented prism is used for directing light from the reflective-display device toward an optical element (the lens <NUM>, which is a collimating-focusing component) so as to collimate the display light, and is also used for directing the incoming light reflected from the eye <NUM> toward the optical element so as to focus the light onto the optical sensor <NUM> via the illumination prism <NUM>. The polarization-and-spectrally selective beamsplitter can be provided via a polarization and spectrally selective coating, in the form of a dielectric coating, directly on one of the hypotenuse sides.

A source of polarized light <NUM> (which can be a combination of a light source (e.g., LED) with a polarizer) is associated with the light-wave entrance surface <NUM>. The source of polarized light <NUM> is configured to emit polarized light in the second optical spectrum (i.e., visible region), represented schematically as incident beam <NUM>. A reflective-display device <NUM> (preferably implemented as a liquid crystal on silicon (LCoS) microdisplay), generating spatial modulation of reflected light corresponding to an image, is associated with the image display surface <NUM>. The reflective-display device <NUM> is illuminated by the incident beam <NUM> from the source of polarized light <NUM> reflected from beam splitter configuration <NUM>. The reflective-display device <NUM> is configured such that the reflected light corresponding to a bright region of a desired image has a polarization rotated relative to the source of polarized light. Thus, as shown in <FIG>, polarized illumination <NUM> enters the prism <NUM> through the light-wave entrance surface <NUM> with a first polarization, typically an s-polarization relative to the surface of the beamsplitter configuration <NUM>, and is reflected towards the image display surface <NUM> where it impinges on the reflective-display device <NUM>. Pixels corresponding to bright regions of the image are reflected with modulated rotated polarization (typically p-polarized) so that radiation from the bright pixels is transmitted through the beamsplitter configuration <NUM> and exits the prism <NUM> via transmission through the light-wave exit-and-entrance surface <NUM>. The light then enters the prism <NUM> through the light-wave exit-and-entrance surface <NUM> with the second polarization (typically p-polarized relative to the surface of the polarization-and-spectrally selective beamsplitter configuration <NUM>) and reaches the collimation-focusing surface <NUM> where it passes through at least one retardation plate <NUM>, preferably a quarter-wave plate, associated with at least part of the collimation-focusing surface <NUM>, enters at least one light-wave collimating-focusing component, namely the lens <NUM>, overlying at least part of the retardation plate <NUM>, and is reflected back through the retardation plate <NUM> by a reflecting surface <NUM> of the lens <NUM>. The double pass through the retardation plate <NUM> aligned with its fast axis at <NUM> degrees to the polarization axes rotates the polarization (e.g., transforming the p-polarization to s-polarization) so that the collimated image illumination is reflected at the polarization-and-spectrally selective beamsplitter configuration <NUM> towards the light-wave exit-and-entrance surface <NUM> and exits the prism <NUM> as the beam of illumination <NUM>. The beam of illumination <NUM> is then coupled into the LOE <NUM> by the optical coupling configuration <NUM>.

<FIG> illustrates schematically the light path followed by the collimated light reflected from the eye <NUM> through the optical module <NUM> after being coupled out of the LOE <NUM> by the optical coupling configuration <NUM>. Recall from <FIG> and <FIG>, the collimated light 116A - 116F reflected from the eye <NUM> that propagates through the LOE <NUM> in the reverse direction is coupled out of the LOE <NUM> by the optical coupling configuration <NUM> as the light rays 122A - 122C representative of the beam of illumination <NUM>. The illumination <NUM> (typically in the NIR region) may include two orthogonally polarized components (i.e., a first polarization component (e.g., p-polarization relative to the surface of the beamsplitter configuration <NUM>) that is collimated and a second polarization component (e.g., s-polarization relative to the surface of the beamsplitter configuration <NUM>) that is not collimated). The illumination <NUM> enters the prism <NUM> through the light-wave exit-and-entrance surface <NUM>. As discussed, the polarization-and-spectrally selective beamsplitter configuration <NUM> reflects p-polarized light and transmits s-polarized light having wavelengths in the first optical spectrum (e.g., NIR region). Therefore, the second polarization component of the illumination <NUM> is transmitted by the beamsplitter configuration <NUM> and exits the prism <NUM> via the surface <NUM>. The first polarization component (typically p-polarization relative to the surface of the beamsplitter configuration <NUM>) of the illumination <NUM> (which is collimated) is reflected by the polarization-and-spectrally selective beamsplitter configuration <NUM> and reaches the collimation-focusing surface <NUM> where it passes through the retardation plate <NUM>, enters the collimating-focusing component <NUM>, and is reflected back through the retardation plate <NUM> by the reflecting surface <NUM> of the lens <NUM>. Whereas the collimating-focusing component <NUM> acts to collimate the uncollimated illumination <NUM> in <FIG>, the collimating-focusing component <NUM> performs the opposite function on the collimated illumination <NUM> in <FIG>, namely applying optical power to the incident collimated light rays so as to convert sets of parallel light rays (collimated light rays 122A, 122B, and 122C) into converging beams of captured light, i.e., focusing the illumination <NUM> on the optical sensor <NUM>. In addition, and similar to as described with reference to <FIG>, the double pass through the retardation plate <NUM> aligned with its fast axis at <NUM> degrees to the polarization axes rotates the polarization (e.g., transforming the p-polarization to s-polarization) of the illumination <NUM> so that the focused illumination is transmitted through the polarization-and-spectrally selective beamsplitter configuration <NUM> and exits the prism <NUM> via transmission through the light-wave exit-and-entrance surface <NUM>. The light then enters the prism <NUM> through the light-wave exit-and-entrance surface <NUM> with a first polarization (typically s-polarization relative to the surface of the beamsplitter configuration <NUM>). As previously discussed, the beamsplitter configuration <NUM> reflects s-polarized light and transmits p-polarized light. These reflection and transmission characteristics are based only on the polarization of the incident light, therefore both NIR and visible light are handled in the same way by the beamsplitter configuration <NUM>. Thus, the s-polarized NIR light is reflected at the beamsplitter configuration <NUM> towards the light-wave exit surface <NUM> and exits the prism <NUM> as a focused beam of illumination <NUM>, which impinges on the optical sensor <NUM>.

It is also noted that for each instance where a particular polarized wave path has been followed in the examples described herein, the polarizations are interchangeable, whereby, for example, on altering the polarization selective properties of the beamsplitter configurations <NUM>, <NUM> and the lens <NUM>, each mention of p-polarized light could be replaced by s-polarized light, and vice versa. For example, the lens <NUM> may be configured to collimate s-polarized components of (NIR) light. In such a configuration, the source of polarized light <NUM> is configured to emit p-polarized incident beam <NUM>, the beamsplitter configuration <NUM> reflects p-polarized light and transmits s-polarized light (in both the photopic and NIR region), and the beamsplitter configuration <NUM> reflects s-polarized light and transmits p-polarized light having wavelengths in the NIR region and reflects p-polarized light and transmits s-polarized light having wavelengths in the photopic (visible light) region.

The polarization-and-spectrally selective beamsplitter configuration <NUM> illustrated in <FIG> and <FIG> may have certain drawbacks, notably the complexity in the design of the coatings that provide the proper spectral-and-polarization-selective transmission and reflection of incident light. One alternative to the beamsplitter design illustrated in <FIG> and <FIG> is shown in <FIG>. Here, the beamsplitter configuration <NUM> is implemented as a polarization selective beamsplitter configuration (similar to the beamsplitter configuration <NUM>), i.e., it reflects s-polarized light and transmits p-polarized light in both the first and second optical spectra (i.e., visible light and NIR light are treated the same by the beamsplitter configuration <NUM>). Since the beamsplitter configuration <NUM> illustrated in <FIG> does not discriminate between light in the first or second optical spectra, two additional retardation plates are deployed to handle polarization rotation for NIR light. Specifically, a retardation plate <NUM> is associated with at least part of the light-wave exit-and-entrance surface <NUM>, and another retardation plate <NUM> is associated with the light-wave exit-and-entrance surface <NUM> and the light-wave exit-and-entrance surface <NUM> so as to be deployed between the prisms <NUM> and <NUM>. The retardation plates <NUM>, <NUM> act as half wave plates for incident light in first optical spectrum (i.e., NIR light) thereby rotating the polarization of incident NIR light, and act as full wave plates for incident light in the second optical spectrum (i.e., photopic (visible) light) thereby not effecting the polarization state of incident photopic light.

Accordingly, the first polarization (typically p-polarized) component of the illumination <NUM> that impinges on the retardation plate <NUM> has its polarization rotated to a second orthogonal polarization (e.g., transforming p-polarization to s-polarization) by the retardation plate <NUM>, and the second polarization (typically s-polarized) component of the illumination <NUM> that impinges on the retardation plate <NUM> has its polarization rotated to the first orthogonal polarization (e.g., transforming s-polarization to p-polarization) by the retardation plate <NUM>. The illumination <NUM> (after having passed through the retardation plate <NUM>) enters the prism <NUM> through the light-wave exit-and-entrance surface <NUM>. The component of the illumination <NUM> that enters the prism as p-polarized (relative to the surface of the polarization selective beamsplitter configuration <NUM>) is transmitted by the beamsplitter configuration <NUM> and exits the prism <NUM> via the surface <NUM>. The component of the illumination <NUM> that enters the prism as s-polarized (relative to the surface of the polarization selective beamsplitter configuration <NUM>) is reflected by the beamsplitter configuration <NUM> and reaches the collimation-focusing surface <NUM> where it passes through the retardation plate <NUM>, enters the collimating-focusing component (i.e., lens) <NUM>, and is reflected back through the retardation plate <NUM> by the reflecting surface <NUM> of the lens <NUM> so as to rotate the polarization (e.g., transforming the s-polarization to p-polarization) so that the focused illumination is transmitted through the polarization selective beamsplitter configuration <NUM> and exits the prism <NUM> via transmission through the light-wave exit-and-entrance surface <NUM>. The light then impinges on the retardation plate <NUM> with a first polarization (typically p-polarized) and has its polarization rotated to a second orthogonal polarization (e.g., transforming p-polarization to s-polarization) by the retardation plate <NUM> such that the illumination <NUM> enters the prism <NUM> through the light-wave exit-and-entrance surface <NUM> with s-polarization relative to the surface of the beamsplitter configuration <NUM>. The now s-polarized light is reflected by the beamsplitter configuration <NUM> towards the light-wave exit surface <NUM> and exits the prism <NUM> as a focused beam of illumination <NUM>, which impinges on the optical sensor <NUM>.

Note that since the retardation plates <NUM>, <NUM> act as full wave plates for photopic light, the path of traversal through the prisms <NUM>, <NUM> from the source of polarized light <NUM> to the output of the prism <NUM> (light-wave exit-and-entrance surface <NUM>), as well as the polarization direction of the traversing light, are unaffected by the retardation plates <NUM>, <NUM>.

It is noted that the configuration of the optical module <NUM> illustrated in <FIG> is applicable to situations in which the lens <NUM> discriminates between a first type of light, i.e., eye tracking light (light from the eye <NUM>) and a second type of light, i.e., image light (light from the reflective-display device <NUM>) based at least in part on polarization separation. In configurations in which the lens <NUM> discriminates between these two types of light based only on spectral separation, the retardation plate <NUM> is not needed. This is due to the fact that the eye may be illuminated such that the eye tracking light generally includes s and p polarization components which are both collimated by the lens <NUM> (since the lens <NUM> collimates light in the optical spectrum occupied by the eye tracking light, e.g., the NIR region without regard to polarization). Thus, the illumination <NUM> that is coupled-out from the LOE <NUM> to the optical module <NUM> is collimated for the components of s-polarization and p-polarization (relative to the surface of the beamsplitter configuration <NUM>). Here, the p-polarization component will enter the prism <NUM> through the surface <NUM>, will be transmitted by the beamsplitter configuration <NUM>, and exit the prism <NUM> through the surface <NUM>. The s-polarization component enters the prism <NUM> through the surface <NUM>, is reflected by the beamsplitter configuration <NUM>, exits the prism <NUM> through the surface <NUM> and reaches the collimation-focusing surface <NUM> where it passes through the retardation plate <NUM>, enters the collimating-focusing component (i.e., lens) <NUM>, and is reflected back through the retardation plate <NUM> by the reflecting surface <NUM> of the lens <NUM> so as to rotate the polarization (e.g., transforming the s-polarization to p-polarization) so that the focused illumination is transmitted through the beamsplitter configuration <NUM> and exits the prism <NUM> via transmission through the light-wave exit-and-entrance surface <NUM>.

Other implementations of the beamsplitter configurations <NUM>, <NUM> are contemplated herein, including, for example, implementation of one or both of the beamsplitters configurations <NUM>, <NUM> of the optical module <NUM> as simple <NUM>-<NUM> beamsplitters, which reflect approximately half of the intensity of incident light and transmit approximately half of the intensity of incident light. Alternatively, both of the beamsplitter configurations can be implemented as polarization selective beamsplitter configurations for incident light in the second optical spectrum (visible light) and as simple <NUM>-<NUM> beamsplitters for incident light in the first optical spectrum (NIR light). For example, the beamsplitter configurations can reflect s-polarized visible light and transmit p-polarized visible light, and reflect approximately half of the intensity of incident NIR light and transmit approximately half of the intensity of incident NIR light. It is noted, however, that in such <NUM>-<NUM> beamsplitter configurations, only approximately <NUM>% of the intensity of the initially incident light reaches the output.

Various configurations of the illumination arrangement <NUM> are contemplated herein. In all of the illumination arrangement configurations, the illumination arrangement <NUM> includes one or more light source configured to illuminate the eye <NUM> with light of the first type (i.e., light in a first optical spectrum (e.g., NIR light) that includes components of light that are polarized in a first polarization direction (e.g., p-polarized)). Ideally, the light source(s) of the illumination arrangement <NUM> is/are deployed to illuminate the eye <NUM> in an illumination direction that is as close to normal to the EMB <NUM> as possible. In an alternative configuration, the light source(s) is/are deployed at periphery of the field of view of the eye <NUM> so as to illuminate the eye <NUM> from the side. In yet another configuration, the illumination arrangement <NUM> is deployed as part of the optical module <NUM>, which in addition to generating and projecting the image <NUM> into the LOE <NUM> for viewing by the eye <NUM> can also be configured to inject light from the illumination arrangement <NUM> into the LOE <NUM> to propagate in the forward direction so as to be coupled out of the LOE <NUM> by the partially reflective surfaces <NUM> in a coupling out direction that is normal to the EMB <NUM>.

The following paragraphs describe several of the deployment options for the illumination arrangement <NUM> with particular reference to <FIG>. The non-limiting implementation of the apparatus <NUM> illustrated in <FIG> is intended to provide context as to the general deployment options for the illumination arrangement <NUM>. In the particular non-limiting implementation illustrated here, the apparatus <NUM> is implemented in an eye-glasses form factor with a head-mounted mechanical body implemented as an eye-glasses frame <NUM> with side arms <NUM> for engaging the ears of the observer. It should be noted that other form factors, such as helmet-mounted form factors, vehicle windshield form factors, and other head-up display and near-eye display form factors also clearly fall within the scope of the present invention. The illumination arrangement <NUM> may include at least one source of light <NUM>A (which in <FIG> is represented as two sources of light) deployed close to the active region of the LOE <NUM> (via for example direct or indirect attachment to the face <NUM>) such that the light rays emitted by the source of light 138A reach the EMB <NUM> close to normal to the EMB <NUM>. Alternatively, or in addition to the source of light <NUM>A, the illumination arrangement <NUM> can include at least one other source of light 138B deployed near the side of the observer's head (in <FIG> this is illustrated as being attached to the optical coupling configuration <NUM> which is attached to the side arm <NUM>). In such a configuration, the light rays emitted by the source of polarized light 138B reach the EMB <NUM> at an off-axis angle. As discussed, the light reflected from the eye <NUM> may include two orthogonally polarized components of light (i.e., s-polarization components and p-polarization components), and the lens <NUM> is configured to collimate only one of these two polarization directions. In the examples described herein, the lens <NUM> is preferably configured to apply optical power to the p-polarized components of light reflected from the eye <NUM> (so as to collimate the p-polarized light) and to apply no optical power to the s-polarized components of light reflected from the eye <NUM>.

The illumination arrangement <NUM> may be configured to illuminate specific regions of the eye <NUM> or the entire eye <NUM> with NIR light. As discussed in detail, the illumination that is reflected by the eye <NUM> (i.e., the light of the first type, represented by the light rays 114A - 114F) is collimated (by the lens <NUM>) and coupled into the LOE <NUM> by the partially reflective surfaces <NUM> and then coupled out of the LOE <NUM> (by the optical coupling configuration <NUM>), where it is focused (by the lens <NUM>) onto the optical sensor <NUM>. The optical sensor <NUM> generates signals in response to sensing the focused light, and those signals are transferred to the processing system <NUM> which is configured to process the signals to derive a current gaze direction of the eye <NUM>. In certain non-limiting implementations, the apparatus <NUM> obtains the gaze direction (the angular orientation of the eye <NUM>, or line of sight of the eye <NUM>) by imaging patterns that exist on specific regions of the eye <NUM>. The position of such patterns and their motion are indicative of the current gaze direction and motion of the eye. The human eye includes various trackable features, including, for example, patterns generated by the blood vessels of the retina. These trackable features can be tracked using appropriate tracking algorithms implemented by suitable image processing instructions performed by the processing system <NUM>.

In a non-limiting process for deriving and tracking the gaze direction, the retina pattern is mapped and trackable features are determined during an initial setup process, and then a continuous tracking process is performed. For example, an image marker may be displayed to the observer for the observer to look at during an initialization. While the observer looks towards the marker, the illumination arrangement <NUM> fully illuminates the fundus (visible portion of the retina) by short pulses and a full image of the fundus obtained (via the optical sensor <NUM>). This image is then processed by processing system <NUM> to identify trackable features (for example, the optic disc and the fovea). During the continuous tracking process, selected regions of interest (ROI) of the eye <NUM> are selectively illuminated by the illumination arrangement <NUM>, and an image of the ROI (obtained by the optical sensor <NUM>) is sampled and processed (by the processing system <NUM>) during the corresponding illumination pulse to determine the current gaze direction (line of sight), and this derived gaze direction is used to update the position of the ROI for the subsequent illumination cycle, and the continuous tracking process repeats by illuminating the updated ROI. Assuming that the frequency of the tracking measurements is high compared to the speed of motion of the eye, this update process is typically effective to maintain continuous tracking, optionally combined with tracking information from the other eye. As the gaze direction changes, so does the illumination area. Updating of the ROI may be performed according to the "current" gaze direction as determined from the last sampled image or, in certain cases, may use predictive extrapolation based on eye motion between the previous two or more measurements. In the event that tracking fails, the size of the illuminated region can be temporarily increased until the trackable features are recovered.

Looking again at <FIG>, the processing system <NUM> may be implemented using any suitable type of processing hardware and/or software, as is known in the art, including but not limited to any combination of various dedicated graphics processors, display drivers, and computerized processors (collectively designated as processor <NUM>) operating under any suitable operating system and implementing suitable software or firmware modules. The storage medium <NUM> can be one or more computerized memory devices, such as volatile data storage. The processing system <NUM> may further include various communications components for allowing wired or wireless communication with LAN and/or WAN devices for bidirectional transfer of information and graphic content. The apparatus <NUM> is powered from a suitable electrical power source, which may be any combination of batteries and/or an external power source provided, illustrated here schematically as power source <NUM> connected via a cable <NUM>. Where battery power is used, the batteries may be integrated as part of the eye-glasses or helmet-mounted structure.

The optical components associated with the faces <NUM>, <NUM> of the LOE <NUM>, such as the lens <NUM> and the polarizer <NUM>, are optically attached to the LOE <NUM> using any suitable attachment technique, including, for example, mechanical attachment to the LOE <NUM> while maintaining an air gap or material (e.g., gel) gap between the optical component and the face of the LOE <NUM>. The material occupying such an air gap or material gap has a refractive index that is sufficiently low enough to preserve the conditions of total internal reflection within the LOE <NUM>. Other suitable alternatives for optically attaching optical components to the LOE <NUM> include deployment of an air gap film having a hyperfine structure between the face of the LOE <NUM> and the optical component, or deployment of a transparent layer of low refractive index material (such as a thin plate of low index material). Further details of such optical attachment methodologies can be found in the applicant's commonly owned <CIT> and <CIT>. The quarter wave plate <NUM> may be attached to the lens <NUM> using similar optical attachment techniques.

Although the embodiments of the apparatus <NUM> described thus far have pertained to an optical coupling configuration implemented as a set of partially reflective surfaces <NUM> for coupling eye tracking light into the LOE <NUM> and for coupling image light (from the optical module <NUM>) out of the LOE <NUM>, the partially reflective surfaces <NUM> are merely illustrative of one non-limiting optical coupling configuration, and other optical coupling configurations can be used to couple eye tracking light into, and image light out of, the LOE <NUM>. The optical coupling configuration may be any optical coupling arrangement which deflects part of the eye tracking incident radiation from the lens <NUM> to an angle which propagates through internal reflection within the LOE <NUM>, and likewise deflects part of the image incident radiation (from the optical module <NUM>) already propagating within the LOE <NUM> by internal reflection to an angle such that the deflected part of the image incident radiation exits the LOE <NUM>. Other examples of such suitable optical coupling arrangements include, but are not limited to, one or more diffractive optical elements deployed on either of the faces <NUM>, <NUM>.

The embodiments of the apparatus as described with respect to <FIG> have pertained to utilization of a polarization and/or spectrally (wavelength) sensitive collimating element (lens <NUM>) to collimate only particular components of the eye-tracking light (i.e., light reflected from the eye of the observer) so as to be able to focus the coupled-in collimated light onto an optical sensor integrated into the image projector (optical module <NUM>). Other eye-tracking solutions are contemplated herein in which uncollimated light from the eye is coupled into a light-guiding optical element and is directed by a bi-conic lens, having a different radius of curvature for two orthogonal axes, onto an optical sensor in order to image the eye and determine the angles at which light emanates from the eye. Such solutions employ a specialized at least partially reflective surface, preferably deployed in a dedicated light-guide optical element separate from the LOE through which the projected image propagates.

Referring now to <FIG>, wherein the examples of <FIG> are not part of the invention, there is illustrated various aspects of the structure and operation of an apparatus, generally designated <NUM>, constructed and operative according to various embodiments of the present invention, for displaying an image and for deriving a gaze direction of the human eye <NUM> by way of a coupling-in configuration associated with a light-guide optical element (LOE). In the preferred but non-limiting implementation illustrated here, the coupling-in configuration is implemented as a surface <NUM> that is at least partially reflective to light emanating from the eye <NUM>. The surface <NUM> is interchangeably referred to hereinafter as an at least partially reflective surface <NUM>. The surface <NUM> is associated with a first LOE <NUM> that is configured for propagating light reflected from the eye (in response to illumination by an illumination arrangement <NUM>), and that is separate from a second LOE <NUM> that is configured for propagating a projected image that is to be coupled out for viewing by the eye <NUM>. The LOE <NUM> is formed from transparent material and has a pair of parallel faces (planar major surfaces) <NUM>, <NUM> for guiding light by internal reflection (preferably total internal reflection). The surface <NUM> is configured for coupling-in a proportion of light incident on the face <NUM> within a coupling-in region <NUM> so to propagate within the LOE <NUM> by (total) internal reflection. In particular, the surface <NUM> is deployed within the LOE <NUM> (i.e., between the faces <NUM>, <NUM>) obliquely to the parallel faces <NUM>, <NUM> such that the coupled-in light is trapped within the substrate <NUM> by internal reflection from the faces <NUM>, <NUM>. The coupling-in region <NUM> of the LOE <NUM>, also referred to as the "active region" or "active area", is a two-dimensional region of the face <NUM>.

The light coupled-in by the surface <NUM> propagates through the LOE <NUM> in the reverse direction until reaching a coupling-out optical configuration <NUM> (shown schematically as a prism in <FIG>, but can also be implemented as, for example, a reflecting surface). Preferably, a mixer <NUM>, implemented as a partially reflective surface, is deployed within the LOE <NUM> on a plane parallel to the faces <NUM>, <NUM> (preferably at the mid-plane between the faces <NUM> and <NUM>) upstream from and adjacent to the coupling out optical configuration <NUM>, to mitigate non-uniformity of the light propagating through the LOE <NUM>. The light is coupled out of the LOE <NUM> to an optical module <NUM> by the coupling-out optical configuration <NUM>. The optical module <NUM> includes a lens <NUM> (a bi-conic lens) and an optical sensor <NUM> (although the sensor <NUM> may be external to the optical module <NUM>). The coupled-out light passes through the lens <NUM> which directs the light onto the optical sensor <NUM> configured for sensing the light reflected from the eye <NUM>. A processing system <NUM>, that includes at least one computerized processor <NUM> coupled to a storage medium <NUM> (such as a computer memory or the like), is electrically associated with the optical sensor <NUM>, and is configured to process signals from the optical sensor <NUM> to derive a current gaze direction of the eye <NUM>.

The second LOE <NUM>, configured for propagating a projected image that is to be coupled out for viewing by the eye <NUM>, is formed from transparent material and has a pair of parallel faces (planar major surfaces) <NUM>, <NUM> for guiding light by internal reflection (preferably total internal reflection). The LOE <NUM> is deployed with one of the parallel faces <NUM> in facing relation to the eye <NUM>, where the eye <NUM> is located in the EMB <NUM> at an eye relief (ER) distance <NUM> from the face <NUM>. An image projector <NUM> is configured to project an image <NUM> (collimated to infinity), as represented here schematically by a beam of illumination <NUM> including sample rays 222A, 222B, and 222C which span the beam. The projected image <NUM> is coupled into the LOE <NUM> by a coupling-in optical configuration <NUM>, as illustrated here schematically by a reflecting surface (but other configurations, such as, for example, prisms, are contemplated herein), so as to generate reflected rays <NUM> which are trapped by internal reflection within the substrate, generating also rays <NUM>. The image propagates along the substrate by repeated internal reflection, impinging on an optical coupling-out configuration associated with the second LOE <NUM>, implemented as a sequence of partially reflecting surfaces <NUM> at an oblique angle to the parallel faces <NUM>, <NUM>, where part of the image intensity is reflected so as to be coupled out of the substrate as rays 226A, 226B, and 226C toward the eye <NUM>. It is noted that the partially reflective surfaces <NUM> are merely illustrative of one non-limiting optical coupling-out configuration suitable for use with the LOE <NUM>, and other optical coupling configurations can be used to couple image light out of the LOE <NUM>. The optical coupling-out configuration may be any optical coupling arrangement which deflects part of the image propagating within the LOE <NUM> by internal reflection to an angle such that the deflected part of the image exits the LOE <NUM>. Other examples of such suitable optical coupling arrangements include, but are not limited to, one or more diffractive optical elements deployed on either of the faces <NUM>, <NUM>.

The LOE <NUM> is deployed with one of the parallel faces <NUM> in facing relation to the eye <NUM>, but with the LOE <NUM> interposed between the eye <NUM> and the LOE <NUM> and such that the faces <NUM> and <NUM> are parallel (or approximately parallel), aligned with, and adjacent to each other. The eye <NUM> is located at an eye relief (ER) distance <NUM> from the face <NUM>. In the non-limiting configuration illustrated in the drawings, the LOEs <NUM> and <NUM> are deployed such that the LOEs <NUM> and <NUM> have a common direction of elongation (illustrated arbitrarily herein as corresponding to the x-axis), and such that the faces <NUM>, <NUM>, <NUM>, <NUM> are mutually parallel. The LOEs <NUM> and <NUM> are preferably optically attached to each other at the faces <NUM>, <NUM> to define an interface plane. Any suitable mechanism may be used for optically attaching the LOEs <NUM> and <NUM> to each other, including but not limited to a mechanical arrangement, and optical cement. For example, the faces <NUM>, <NUM> may be cemented to each other by providing a layer of optical cement to at least one portion of at least one of the faces <NUM>, <NUM> to form a cemented unitary optical structure formed from two light guides that perform separate functions.

As in the embodiments described with reference to <FIG>, in the present embodiments the eye <NUM> is preferably illuminated with light in a first optical spectrum (preferably the NIR region) such that light in the first optical spectrum is coupled into the LOE <NUM> by the surface <NUM>, and the illumination <NUM> (i.e., the projected image) is in a second optical spectrum (the photopic, i.e., visible region). As discussed in the previous embodiments, the partially reflective surfaces <NUM> are preferably coated so as to have low reflectance for a first range of incident angles, while having the desired partial reflectivity for a second range of incident angles. In addition, the faces <NUM>, <NUM> and the partially reflective surfaces <NUM> are preferably coated so as to have high transmittance for light in the first optical spectrum, such that the light reflected by the eye <NUM> passes through the LOE <NUM> with minimal loss of intensity before being coupled into the LOE <NUM>.

In contrast to the incident light from the eye that is coupled into the LOE <NUM> in the previously described embodiments (<FIG>), in the present embodiments the incident light from the eye <NUM> that is coupled into the LOE <NUM> is not collimated, resulting in an angular distribution of rays of incident light impinging on the surface <NUM> at different respective incident angles. With particular reference to <FIG>, incident light from two different points within the EMB <NUM> along a first dimension of the EMB <NUM> (the first dimension being along the x-axis in the arbitrarily labeled XYZ coordinate system in the drawings) impinge on the surface <NUM> so as to be coupled into the LOE <NUM>. The incident light, represented here schematically by a first beam of illumination <NUM> and a second beam of illumination <NUM>, is light (preferably in the NIR region) that is reflected by the eye <NUM> in response to illumination from the illumination arrangement <NUM>. Note that the two beams <NUM>, <NUM> merely illustrate a sample of the beams from the EMB <NUM> that are coupled into the LOE <NUM> by the surface <NUM>, additional beams from additional respective points within the EMB <NUM> are also coupled into the LOE <NUM> by the surface <NUM>. As can be seen, each of the beams <NUM>, <NUM> generally arrives at the surface <NUM> at a different incident angle, such that the beams <NUM>, <NUM> generate respective reflected beams <NUM>, <NUM> that are trapped within the LOE <NUM> by internal reflection, but which propagate within the LOE <NUM> at different angles relative to the faces <NUM>, <NUM>.

As shown in <FIG>, each of the beams <NUM>, <NUM> includes light rays which span the beam. In the illustrated example, the beam <NUM> includes sample rays 246A, 246B, and 246C which span the beam <NUM> along at least a portion of the coupling-in region <NUM> in the first dimension (along the x-axis), where the rays 246A and 246C are the marginal rays of the beam <NUM>. Similarly, the beam <NUM> includes sample rays 252A, 252B, and 252C along at least a portion of the coupling-in region <NUM> in the first dimension (along the x-axis), where the rays 252A and 252C are the marginal rays of the beam <NUM>. Each of the rays 246A, 246B, 246C, 252A, 252B, and 252C is incident on the face <NUM> at a different respective point of the face <NUM> along the first dimension of the face <NUM> (the x-axis in the drawings), and therefore is incident on the surface <NUM> at different respective incident angles. Therefore, each of the rays 246A, 246B, 246C, 252A, 252B, and 252C arrives at the surface <NUM> at a different incident angle, such that the reflected beams <NUM>, <NUM> each include spaced apart reflected rays (spanning the respective beam) which propagate within the LOE <NUM>.

The angular distribution of the light (spanned by the beams <NUM>, <NUM>, and spanned by the rays spanning the beams <NUM>, <NUM>) that is coupled into the LOE <NUM> by the surface <NUM> is a function the aperture width of the surface <NUM> (the width being projected on the plane parallel to the EMB <NUM>). The aperture width is inversely proportional to the steepness of the deployment angle β of the surface <NUM> (measured relative to the face <NUM>), such that for steep deployment angles, the aperture width is effectively small, thereby providing high resolution in the angular spanning dimension (x-axis in the drawings). In the present embodiments, the surface <NUM> is deployed at a steeper angle than the partially reflective surfaces <NUM>, and is deployed at an angle steep enough such that the aperture width of the surface <NUM> is narrow enough relative to the distance between the LOE <NUM> and the EMB <NUM> such that light covering only a narrow angular distribution of angles is coupled into the LOE <NUM> by the surface <NUM>.

The resolution can roughly be approximated by the width of the surface <NUM> projected on the plane parallel to the face <NUM>. In <FIG>, the width is denoted as w, and can be calculated as h/tan(β), where h is thickness of the LOE <NUM> (i.e., the minimum distance between the planar faces <NUM>, <NUM>). For example, for h = <NUM> and β = <NUM>°, w ≈ <NUM>. A resolution of <NUM> is smaller than that of the pupil of the human eye, so such parameters for h and β could provide a high-resolution image at the optical sensor <NUM>, and decreasing the width (by increasing the deployment angle β and/or decreasing the thickness h) could produce an even higher resolution image. It is noted, however, that as the width is decreased, the strength of the signals output by the optical sensor <NUM> also decrease, reducing the overall signal-to-noise ratio of the output signal. Therefore, care should be taken to find the appropriate balance between a small aperture width that corresponds to a reasonable signal-to-noise ratio at the optical sensor <NUM>.

Turning now to <FIG>, there is shown coupled-out light rays 247A, 247B, and 247C, and the coupled-out light rays 253A, 253B, and 253C, being received at the optical module <NUM>. The light rays 247A, 247B, and 247C and the light rays 253A, 253B, and 253C correspond to incident light rays spanning an angular distribution in a first dimension (width) of the surface <NUM>. In particular, the coupled-out light rays 247A, 247B, and 247C correspond to the incident light rays 246A, 246B, and 246C, and the coupled-out light rays 253A, 253B, and 253C correspond to the incident light rays 252A, 252B, and 252C. The coupled-out light rays 247A, 247B, and 247C, and the coupled-out light rays 253A, 253B, and 253C pass through the lens <NUM>, which applies optical power to the light rays so as to direct the light rays to the optical sensor <NUM>.

As mentioned, the lens <NUM> is bi-conic, which in the present context refers to having a different radius of curvature for different axes. The different radii of curvature lead to the lens <NUM> having two focal lengths in two respective dimensions (orthogonal dimensions), namely a first focal length of f<NUM> in a first dimension and a second focal length of f<NUM> in a second dimension (orthogonal to the first dimension). The optical sensor <NUM> is deployed at a distance of f<NUM> from the lens <NUM>. The first focal length f<NUM> and the positioning of optical sensor <NUM> at the first focal length f<NUM> are such that the lens <NUM> converts the light rays 247A, 247B, and 247C (and the light rays 253A, 253B, and 253C) into non-converging beams of captured light that reach different respective regions of the optical sensor <NUM> such that the angular distribution of the light rays 247A, 247B, and 247C (and the light rays 253A, 253B, and 253C) is indicative of the angular distribution (in the width dimension of the surface <NUM>) of the corresponding beam <NUM> and (beam <NUM>). Furthermore, the light rays 247A, 247B, and 247C, and the light rays 253A, 253B, and 253C reach different respective regions of the optical sensor <NUM> such that the overall angular separation between the sets of the rays 247A, 247B, and 247C, and the light rays 253A, 253B, and 253C at the optical sensor <NUM> is indicative of the angular separation (in the width dimension of the surface <NUM>) between the beams <NUM> and <NUM>. The optical sensor <NUM> can therefore measure the relative angles of the light (beams <NUM>, <NUM>) emanating from the EMB <NUM> that is coupled into the LOE <NUM> by the surface <NUM> with a reasonably high angular resolution. The angular resolution is generally a function of the effective aperture width of the surface <NUM> (previously described) and the eye relief (ER <NUM>), and can be expressed as sin-<NUM>(w/ER). For an effective aperture width (w) of <NUM> and an eye relief (ER) of <NUM>, the angular resolution provided by the optical sensor <NUM> is approximately <NUM> degree. Parenthetically, as a result of the small angular resolution, the requirements for parallelism between the principle planes of the LOE <NUM> is much more lenient than for the LOE <NUM> used for image projection to the eye <NUM>, where parallelism on the order of about <NUM> arcmin may be required.

The lens <NUM> has a second focal length f<NUM> in a dimension orthogonal to the first focal length dimension. The bi-conic aspect of the lens <NUM> enables imaging of the eye <NUM> via incident light (reflected from the eye <NUM>) spanning the coupling-in region <NUM> along two orthogonal dimensions. The imaging, via directing (by the lens <NUM>) the coupled-out light corresponding to the incident light rays spanning the first dimension (along the x-axis) was discussed with reference to <FIG>. The following paragraphs will describe the imaging of the eye <NUM> via focusing coupled-out light rays, corresponding to incident light rays spanning a second dimension (along the z-axis), by the lens <NUM> onto the optical sensor <NUM>.

Referring now to <FIG>, the beams <NUM>, <NUM> also include light rays which span the respective beams along at least a portion of the coupling-in region <NUM> in the second dimension (along the z-axis). In the illustrated example, the beam <NUM> includes sample rays 248A, 248B, and 248C which originate from a common point of the EMB <NUM> and span the beam <NUM> along at least a portion of the coupling-in region <NUM> in the second dimension (along the x-axis), where the rays 248A and 248C are the marginal rays of the beam <NUM>. Similarly, the beam <NUM> includes sample rays 254A, 254B, and 254C which originate from a common point of the EMB <NUM> and span the beam <NUM> along at least a portion of the coupling-in region <NUM> in the second dimension (along the x-axis), where the rays 254A and 254C are the marginal rays of the beam <NUM>. The projection of the rays 246A, 246B, and 246C in the XY-plane are incident on the surface <NUM> at a common incident angle. Therefore, the light rays 246A, 246B, and 246C, when coupled-into the LOE <NUM> by the surface <NUM> generate sets of spaced apart parallel rays that propagate through the LOE <NUM>. Similarly, the projection of the rays 254A, 254B, and 254C in the XY-plane are incident on the surface <NUM> at a common incident angle. Therefore, the light rays 254A, 254B, and 254C, when coupled-into the LOE <NUM> by the surface <NUM> generate sets of spaced apart parallel rays that propagate through the LOE <NUM>.

Turning now to <FIG>, there is shown coupled-out light rays 249A, 249B, and 249C, and the coupled-out light rays 255A, 255B, and 255C, being received at the optical module <NUM>. The light rays 249A, 249B, and 249C and the light rays 255A, 255B, and 255C correspond to incident light rays spanning the first dimension (height, along the z-axis) of the surface <NUM>. In particular, the coupled-out light rays 249A, 249B, and 249C correspond to the incident light rays 248A, 248B, and 248C, and the coupled-out light rays 255A, 255B, and 255C correspond to the incident light rays 254A, 254B, and 254C. The coupled-out light rays 249A, 249B, and 249C, and the coupled-out light rays 255A, 255B, and 255C, pass through the lens <NUM>, which applies optical power to the light rays so as to focus the light rays 249A, 249B, and 249C onto a common region (or spot) on an image plane of the optical sensor <NUM>, and to focus the light rays 255A, 255B, and 255C onto a different common spot on the image plane of the optical sensor <NUM>. In other words, the lens <NUM> converts the set of light rays 249A, 249B, and 249C into converging beams of captured light, and likewise converts the set of light rays 255A, 255B, and 255C into converging beams of captured light. The ability to focus these sets of the light rays onto the image plane is enabled by positioning the lens <NUM> at the output aperture of the LOE <NUM> and designing the lens <NUM> with a suitable second focal lengthf<NUM>.

In general, the lens <NUM> is preferably designed such that the second focal length f<NUM> is given by f<NUM> = u f<NUM>/(u - f<NUM>), where u is the in-plane distance light rays travel from the surface <NUM> to the lens <NUM> along the second dimension, and can be given by u = ER + L<NUM>/cos(θ), where L<NUM> is the in-plane distance from the surface <NUM> to the coupling-out optical configuration <NUM>, and θ is the angle (measured relative to the face <NUM>) at which the light propagates.

As in the embodiments described with reference to <FIG>, the optical sensor <NUM> in the present embodiments generates signals in response to sensing the light rays that reach the sensor, and those signals are transferred to the processing system <NUM> which is configured to process the signals to derive a current gaze direction of the eye <NUM>. The derivation of gaze direction may be performed using similar steps to those previously described with reference to <FIG>. In addition, the capability of the optical sensor <NUM> to measure the relative angles of the incident light (beams <NUM>, <NUM>) can be used to bolster the derivation of gaze direction in the present embodiments.

The possible deployment configurations of the illumination arrangement <NUM> are generally similar to those of the illumination arrangement <NUM> described with reference to <FIG>. For example, the apparatus <NUM> may be implemented in an eye-glasses form factor with a head-mounted mechanical body implemented as an eye-glasses frame with side arms for engaging the ears of the observer. Other form factors, such as helmet-mounted form factors, vehicle windshield form factors, and other head-up display and near-eye display form factors are also contemplated herein. The illumination arrangement <NUM> may include one or more source of NIR light, which can be deployed, for example, close to the active region of the LOE <NUM> via for example direct or indirect attachment to the face <NUM> such that the light rays emitted by the source of light reach the EMB <NUM> close to normal to the EMB <NUM>. Alternatively, or in addition to the aforementioned configuration, the illumination arrangement <NUM> can include at least one other source of NIR light deployed near the side of the observer's head, for example attached to the image projector <NUM> or the coupling-in optical configuration <NUM> (which are preferably attached to one of the side arms of the eye-glasses frame). In such a configuration, the light rays emitted by the source of NIR light reach the EMB <NUM> at an off-axis angle.

In addition, the LOE <NUM> may be used to illuminate the eye <NUM> at directions normal to the EMB <NUM>. In such a configuration, the illumination arrangement <NUM> is integrated as part of the image projector <NUM>, as illustrated in <FIG>. The image projector <NUM> is generally similar to the image projector <NUM> illustrated in <FIG>, with the exception that the image projector <NUM> does not include the optical sensor (since the optical sensor <NUM> is deployed in the optical module <NUM> that is separate optical from the image projector <NUM>). In addition, since the optical sensor <NUM> is not part of the image projector <NUM>, there is no need to design the beamsplitter configurations <NUM> and <NUM> with suitable coatings for spectral selectivity. Therefore, in the non-limiting example of the image projector <NUM> illustrated in <FIG>, both of the beamsplitter configurations <NUM> and <NUM> are polarization selective beamsplitters which reflect incident light in a first polarization direction (e.g., s-polarized relative to the surface of the beamsplitter configuration <NUM>, <NUM>) and transmit incident light in a second polarization direction (e.g., p-polarized relative to the surface of the beamsplitter configuration <NUM>, <NUM>) for incident light in the first optical spectrum and incident light in the second optical spectrum (i.e., visible light and NIR light are treated the same by the beamsplitter configurations <NUM> and <NUM>). In addition, since light only propagates through the LOE <NUM> in the forward direction, and therefore light does not enter the image projector <NUM> from the LOE <NUM>, the surface <NUM> is a light-wave exit surface <NUM>, the surface <NUM> is a light-wave entrance surface <NUM>, and the surface <NUM> is a light-wave exit surface <NUM>.

Similar to as described with reference to <FIG>, the source of polarized light <NUM> emit polarized light in the second optical spectrum (i.e., visible region), represented schematically as incident beam <NUM>. The polarized illumination <NUM> enters the prism <NUM> through the light-wave entrance surface <NUM> with a first polarization, typically an s-polarization relative to the surface of the polarization selective beamsplitter configuration <NUM>, and is reflected towards the image display surface <NUM> by the polarization selective beamsplitter configuration <NUM> where it impinges on the reflective-display device <NUM>. Pixels corresponding to bright regions of the image are reflected with modulated rotated polarization (typically p-polarized) so that radiation from the bright pixels is transmitted through the beamsplitter configuration <NUM> and exits the prism <NUM> via transmission through the light-wave exit surface <NUM>. The light then enters the prism <NUM> through the light-wave entrance surface <NUM> with the second polarization (typically p-polarized relative to the surface of the polarization selective beamsplitter configuration <NUM>) and reaches the collimation surface <NUM> where it passes through the retardation plate <NUM>, enters the lens <NUM>, and is reflected back through the retardation plate <NUM> by the reflecting surface <NUM> of the lens <NUM>. The double pass through the retardation plate <NUM> aligned with its fast axis at <NUM> degrees to the polarization axes rotates the polarization (e.g., transforming the p-polarization to s-polarization) so that the collimated image illumination is reflected at the polarization selective beamsplitter configuration <NUM> towards the light-wave exit surface <NUM> and exits the prism <NUM> as the beam of illumination <NUM>. The beam of illumination <NUM> is then coupled into the LOE <NUM> by the coupling-in optical configuration <NUM> (as illustrated in <FIG>).

The illumination arrangement <NUM>, implemented, for example, as a source of polarized NIR light (which can be a combination of a NIR light source with a polarizer) is associated with the surface <NUM>, which in this configuration is a light-wave entrance surface <NUM>. The source of polarized NIR light is configured to emit polarized light in the first optical spectrum (i.e., polarized NIR light), represented schematically as incident beam <NUM>. The polarized illumination <NUM> enters the prism <NUM> through the light-wave entrance surface <NUM> with a first polarization, typically a p-polarization relative to the surface of the polarization selective beamsplitter configuration <NUM>, and is transmitted through the polarization selective beamsplitter configuration <NUM> and exits the prism <NUM> via transmission through the light-wave exit surface <NUM> as a beam of illumination <NUM>. The p-polarized beam of illumination <NUM> is then coupled into the LOE <NUM> by the coupling-in optical configuration <NUM> (similar to as the beam of illumination <NUM>). The p-polarized illumination <NUM> propagates through the LOE <NUM> (similar to the illumination <NUM>), and is coupled out of the LOE <NUM> by the partially reflective surfaces <NUM>. In this configuration, care should be taken to ensure that the NIR illumination propagating within the LOE <NUM> is coupled-out by the partially reflective surfaces <NUM> and that the NIR light emanating from the eye <NUM> (in response to illumination by the coupled-out NIR light) is not coupled back into the LOE. To this end, the partially reflective surfaces <NUM> are preferably coated such they have the desired reflectivity for s-polarized light in the NIR region at a prescribed range of incident angles such that the s-polarized NIR illumination propagating within the LOE <NUM> is coupled-out by the partially reflective surfaces <NUM> but the s-polarized NIR illumination emanating from the eye <NUM> is incident on the partially reflective surfaces <NUM> at incident angles outside of the prescribed range of incident angles and therefore passes through the partially reflective surfaces without reflection.

Although the embodiments of the apparatus <NUM> have thus far been described within the context of the LOEs <NUM> and <NUM> having a common (parallel) direction of elongation, other embodiments are possible in which the LOEs have directions of elongation which are orthogonal to each other. For example, the LOE <NUM> may be deployed so as to have a direction of elongation in the direction of the x-axis (as shown in <FIG>), whereas the LOE <NUM> may be deployed so as to have a direction of elongation in the direction of the z-axis. In addition, it is noted that the embodiments of the apparatus <NUM> and <NUM> have been described within the context of LOEs <NUM> and <NUM> being "one-dimensional waveguides" or "1D waveguides", meaning that that LOEs <NUM> and <NUM> each have a single pair of parallel major surfaces (faces <NUM>, <NUM> and faces <NUM>, <NUM>) defining a "slab-type waveguide" which guides image light (from an image projector <NUM>, <NUM>) so as to perform aperture expansion in one dimension. However, the eye-tracking apparatus according to the present embodiments are equally applicable to other waveguide constructions, including constructions in which an additional slab-type waveguide is coupled to each of the LOEs <NUM>, <NUM> which guides image light in an orthogonal dimension so as to perform aperture expansion in the orthogonal dimension, producing an overall two-dimensional aperture expansion effect. Alternatively, one or both of the LOEs <NUM>, <NUM> is a "two-dimensional waveguide" or "2D waveguide", meaning that it has two mutually orthogonal pairs of major surfaces which serve to guide image light (from an image projector <NUM>, <NUM>) in two dimensions as it propagates along the LOE so as to perform aperture expansion in two dimensions using a single waveguide.

Although the embodiments of the present disclosure have been described within the context of illumination arrangements deployed to illuminate the eye with light in the near infrared region of the electromagnetic spectrum, the embodiments of the present disclosure should not be limited to illumination arrangements that emit eye-tracking light in any specific region of the electromagnetic spectrum. The description of using NIR light for eye-tracking purposes is for example purposes in order to provide a clearer explanation of the construction and operation of the various apparatus of the present disclosure. Other types of light may also be used for eye-tracking purposes, including, but not limited to, light in the infrared region, and ultraviolet light emitted at low intensity and short pulse duration.

According to certain non-limiting implementations, the various eye-tracking apparatus of the present disclosure may be duplicated for tracking both eyes of a subject simultaneously, as well as for projecting images to both eyes. For example, the apparatus <NUM> and/or the apparatus <NUM> may be duplicated for both eyes. By combining data from two eye trackers, it may be possible to achieve enhanced stability and continuity of tracking. For example, while the eyes are moving, the trackable portions of the eyes may be visible to the tracker in one eye and not the other. If a tracking algorithm is used which employs tracking of trackable features, simultaneous tracking for both eyes allows the tracking to be maintained continuously through periods in which only one eye-tracker can track the blind spot.

Where an apparatus is binocular, each eye has its own image projection and eye tracking device, and various processing and power-supply components may optionally be shared between the two eye-tracking systems. The eye-tracking information gleaned by the binocular eye-tracking devices can be fused in order to provide enhanced stability and continuity of tracking, as mentioned above.

The descriptions of the various embodiments of the present disclosure have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of protection which is defined by the appended claims. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

As used herein, the singular form, "a", "an" and "the" include plural references unless the context clearly dictates otherwise.

To the extent that the appended claims have been drafted without multiple dependencies, this has been done only to accommodate formal requirements in jurisdictions which do not allow such multiple dependencies. It should be noted that all possible combinations of features which would be implied by rendering the claims multiply dependent are explicitly envisaged and should be considered part of the invention.

Claim 1:
An apparatus (<NUM>), comprising:
a light-transmitting substrate (<NUM>) having at least two parallel major surfaces (<NUM>, <NUM>) for guiding light by internal reflection, a first of the major surfaces (<NUM>) being deployed in facing relation to an eye (<NUM>);
an optical element (<NUM>) associated with the first of the major surfaces (<NUM>), the optical element (<NUM>) configured for applying optical power to incident light in accordance with at least one property of the incident light, wherein the at least one property is the wavelength, such that the optical element (<NUM>) applies non-zero optical power to incident light of a first type returned from the eye (<NUM>) within a first not visible optical spectrum so as to collimate the incident light of the first type and such that the optical element (<NUM>) applies substantially zero optical power to incident light of a second type within a second visible optical spectrum;
an optical coupling configuration (<NUM>) associated with the substrate (<NUM>) and configured for:
coupling-in a proportion of light of the first type, collimated by the optical element (<NUM>) and incident on the first of the major surfaces (<NUM>), so as to propagate within the substrate (<NUM>) by total internal reflection, and
coupling-out a proportion of light of the second type propagating within the substrate (<NUM>);
optics (<NUM>) associated with the substrate (<NUM>) and configured for converting the collimated light of the first type into converging beams of captured light;
an optical sensor (<NUM>) deployed for sensing the captured light; and
at least one processor (<NUM>, <NUM>) electrically associated with the optical sensor (<NUM>) and configured to process signals from the optical sensor (<NUM>) to derive a current gaze direction of the eye (<NUM>)
wherein the optical element (<NUM>) is positioned between the light-transmitting substrate (<NUM>) and the eye (<NUM>).