EFFICIENT MODELING OF A DIFFRACTIVE WAVEGUIDE

A method of simulating the optical performance of a diffractive waveguide includes, generating a plurality of transfer matrices for each diffractive grating of the plurality of diffractive gratings responsive to performing a diffraction modeling process for a plurality of diffractive gratings of the waveguide based on a plurality of input light rays each having at least a different first characteristic. A plurality of electric fields at outcoupling positions of an outcoupling grating of the plurality of diffractive gratings is determined based on the plurality of transfer matrices responsive to performing a ray tracing process for multiple instances of each input light ray of the plurality of light rays with at least a different second characteristic. A uniformity map is generated for the waveguide based on the plurality of electric fields. The uniformity map indicates a uniformity of one or more characteristics of the waveguide across different sampled pupil positions.

BACKGROUND

Near-to-eye display (NED) devices (e.g., augmented reality glasses, mixed reality glasses, virtual reality headsets, and the like) are wearable electronic devices that combine real-world and virtual images via one or more optical combiners, such as one or more integrated combiner lenses, to provide a virtual display that is viewable by a user when the wearable display device is worn on the head of the user. One class of optical combiner uses a waveguide (also termed a lightguide) to transfer light. In general, light from a projector of the wearable display device enters the waveguide of the optical combiner through an incoupler, propagates along the waveguide, and exits the waveguide through an outcoupler. If the pupil of the eye is aligned with one or more exit pupils provided by the outcoupler, at least a portion of the light exiting through the outcoupler will enter the pupil of the eye, thereby enabling the user to see a virtual image. Since the combiner lens is transparent, the user will also be able to see the real world.

SUMMARY OF EMBODIMENTS

In accordance with one aspect, a method includes generating a plurality of transfer matrices for each diffractive grating of the plurality of diffractive gratings and background areas of the waveguide responsive to performing a diffraction modeling process for a plurality of diffractive gratings of a waveguide based on a plurality of input light rays each having at least a different first characteristic A plurality of electric fields at outcoupling positions of an outcoupling grating of the plurality of diffractive gratings is determined based on the plurality of transfer matrices responsive to performing a ray tracing process for multiple instances of each input light ray of the plurality of input light rays with at least a different second characteristic. A uniformity map for the waveguide is generated based on the plurality of electric fields, the uniformity map indicating a uniformity of one or more characteristics of the waveguide across different sampled pupil positions.

In accordance with another aspect, a processing system includes a processor and a waveguide modeler. The waveguide modeler is configured by the processor to generate a plurality of transfer matrices for each diffractive grating of the plurality of diffractive gratings and background areas of the waveguide responsive to a diffraction modeling process performed for a plurality of diffractive gratings of a waveguide based on a plurality of input light rays each having at least a different first characteristic. The waveguide modeler is further configured by the processor to determine a plurality of electric fields at outcoupling positions of an outcoupling grating of the plurality of diffractive gratings based on the plurality of transfer matrices responsive to a ray tracing process performed for multiple instances of each input light ray of the plurality of input light rays with at least a different second characteristic. The waveguide modeler is further configured by the processor to generate a uniformity map for the waveguide based on the plurality of electric fields, the uniformity map indicating a uniformity of one or more characteristics of the waveguide across different sampled pupil positions.

In accordance with a further aspect, a wearable head-mounted display system includes an image source to project light comprising an image, at least one lens element, and waveguide. The waveguide is designed by a process including generating a plurality of transfer matrices for each diffractive grating of the plurality of diffractive gratings and background areas of the waveguide responsive to performing a diffraction modeling process for a plurality of diffractive gratings of a waveguide based on a plurality of input light rays each having at least a different first characteristic. A plurality of electric fields at outcoupling positions of an outcoupling grating of the plurality of diffractive gratings is determined based on the plurality of transfer matrices responsive to performing a ray tracing process for multiple instances of each input light ray of the plurality of input light rays with at least a different second characteristic. A uniformity map for the waveguide is generated based on the plurality of electric fields, the uniformity map indicating a uniformity of one or more characteristics of the waveguide across different sampled pupil positions.

DETAILED DESCRIPTION

Prior to being implemented in NEDs and other devices, waveguides need to be designed and fabricated. A waveguide design is typically modeled so that various aspects of the design can be examined to ensure optimal performance of the waveguide. For example, light propagation, modes of propagation, waveguide geometry, material properties, coupling efficiency, and other aspects of the waveguide design are modeled. By modeling these waveguide aspects, designers can predict the waveguide's performance, optimize the design of the waveguide, and assess how changes to the waveguide might impact the NED's overall performance.

However, modeling a waveguide, such as a diffractive waveguide, is a challenging problem since two drastically different optics regimes, i.e., diffractive optics and refractive optics, typically need to be bridged. Most current modeling techniques and tools are configured to either model diffractive optics or refractive optics, but not both. For example, diffractive waveguides typically implement nanometer-scale diffractive gratings and a macroscopic waveguide structure. The nanoscale diffractive gratings are typically modeled using, for example, either Rigorous Coupled-Wave Analysis (RCWA) or Finite-Different Time-Domain (FDTD). In contrast, light ray propagation inside the macroscopic waveguide structure is typically modeled using a computational tool referred to as a “ray tracer”, which models the effects of the waveguide refractive optics as light rays propagate through the waveguide. Current modeling techniques and tools are not able to effectively integrate the two disparate types of modeling so that diffractive waveguides can be efficiently and accurately modeled.

As such, the following describes embodiments of systems and methods for efficiently and accurately simulating the optical performance of a diffractive waveguide by integrating both diffractive grating modeling to model the wave optics effect and ray tracing to model the refractive optics effect. As described in greater detail below, a waveguide modeler simulates a plurality of input light rays, each having a different wavelength or range of wavelengths (e.g., red, green, or blue wavelengths). Each of these input light rays is simulated multiple times, with each simulation projecting the input light ray onto an input coupler (IC) grating of the waveguide at different field-of-view (FOV) angle (also referred to herein as “field angles” or “incident angles”). Each input light ray simulated with a different field angle is evaluated for multiple different incident ray positions on the IC grating of the waveguide to determine the optical performance of the waveguide. As such, in at least some embodiments, a light ray simulation refers to simulating a light ray having a specified wavelength that is projected with a specified field angle onto the IC of the waveguide at a specified incident position.

For example, given a simulated input light ray having a specified wavelength(s) and a specified field angle, the waveguide modeler performs a diffraction modeling process for each grating structure, such as the IC grating, an exit pupil expander (EPE) grating, and an outcoupler (OC) grating, of the waveguide. The diffraction modeling process executes one or more computational techniques, such as RCWA, on each grating structure to generate a transfer matrix for each interaction between the light ray and the grating. Stated differently, the diffraction modeling process generates transfer matrices between the input light rays and the light rays diffracted by the grating structure.

The waveguide modeler then performs ray tracing for the simulated input light ray. For example, the waveguide modeler selects an incident ray position on the waveguide and, based on the selected incident ray position, generates a grid of nodes where the light ray intersects the waveguide and grating surfaces. Each bounce the light ray makes on the waveguide or grating surface is a node in the grid of nodes. In at least some embodiments, the positions of the nodes are determined from a k-space diagram generated for the waveguide. The waveguide modeler recursively calculates the near-field electric field (also referred to herein as “E-field”) of each node in the grid of nodes using transfer matrices generated during the diffraction modeling process. The transfer matrices relate the E-field of one node to its neighboring nodes. As such, the E-field of each node in the grid of nodes is determined by recursively performing matrix multiplication using one or more transfer matrices generated for the grating structures of the waveguide and neighboring E-fields. The output of an iteration of the ray tracing process, in at least some embodiments, is the E-field associated with each node in the OC of the waveguide. Stated differently, the output of an iteration of the ray tracing process is a set of output E-fields comprised of the E-field for each light ray outcoupled by the OC of the waveguide.

The waveguide modeler then simulates the input light ray with a different field angle and performs the diffraction modeling process described above to generate a new set of transfer matrices. The waveguide modeler repeats the ray-tracing process using the new set of transfer matrices to generate the output E-fields for the current simulated input light ray. The diffraction modeling process and ray-tracing process are repeated until the input light ray having the specified wavelength(s) has been simulated with each remaining field angle of interest. The waveguide modeler also performs the diffraction modeling process and ray-tracing process for additional input light rays simulated with different wavelengths or ranges of wavelengths. For example, if the first simulated input light had a red wavelength range, the waveguide modeler repeats the diffraction modeling and ray-tracing processes for a simulated input light ray having a blue wavelength range and a simulated input light ray having a green wavelength range.

Given the output E-fields for all the field angles of the different input light rays, the waveguide modeler generates a pupil efficiency uniformity map (also referred to herein as “far-field uniformity map”). The pupil efficiency uniformity map indicates the efficiency of the waveguide. For example, the pupil efficiency uniformity map indicates, for different sampled pupil positions and field angles, the brightness of the outcoupled light rays, the color and brightness uniformity of the outcoupled light rays, and the like. This information can be used to determine if the overall waveguide or individual components of the waveguide are performing according to design specifications. Also, in at least some embodiments, the pupil efficiency uniformity map is used to adjust the design parameters of the waveguide and its components, such as the grating structures, to increase the performance of the waveguide.

FIG.1illustrates a block diagram of an example processing system in which the waveguide modeling techniques described herein can be implemented. It should be understood that the techniques described herein are not limited to the processing system100shown inFIG.1. In at least some embodiments, the processing system100includes, for example, a server, a desktop computer, a laptop/notebook, a mobile device, a tablet computing device, a wearable computing device, or the like. The processing system100, in at least some embodiments, comprises a processor102, memory104, storage106, one or more input devices108, and one or more output devices110. The processing system100, in at least some embodiments, also comprises one or more of an input driver112or an output driver114. In some embodiments, the processing system100includes one or more software, hardware, circuitry, and firmware components in addition to or different from those shown inFIG.1.

In at least some embodiments, the processor102comprises a central processing unit (CPU), an accelerator processor (e.g., a graphics processing unit (GPU)), a CPU and an accelerator processor located on the same die or multiple dies (e.g., using a multi-chip-module (MCM)), or one or more processor cores, wherein each processor core is a CPU or an accelerator processor. The memory104, in at least some embodiments, is located on the same die as the processor102or is located separately from the processor102. The memory104includes a volatile or non-volatile memory, such as random-access memory (RAM), dynamic RAM, cache, and so on.

The storage106, in at least some embodiments, comprises a fixed or removable storage, such as a hard disk drive, a solid-state drive, an optical disk, a flash drive, and so on. In at least some embodiments, the input devices108comprise, for example, one or more of a keyboard, a keypad, a touch screen, a touchpad, a detector, a microphone, an accelerometer, a gyroscope, a biometric scanner, a network connection (e.g., a wireless local area network card for transmission/reception of wireless signals), and so on. The output devices110, in at least some embodiments, comprise, for example, one or more of a display, a speaker, a printer, a haptic feedback device, one or more lights, an antenna, or a network connection (e.g., a wireless local area network card for transmission/reception of wireless signals), and so on.

In at least some embodiments, the input driver112communicates with the processor102and the input devices108and allows the processor102to receive input from the input devices108. The output driver114, in at least some embodiments, communicates with the processor102and the output devices110and allows the processor102to send output to the output devices110. It is noted that the processing system100operates in the same manner if the input driver112and the output driver114are not present. The output driver114, in at least some embodiments, includes an accelerated processing device (APD)116that is coupled to a display device118. The APD116accepts compute commands and graphics rendering commands from processor102, processes those compute and graphics rendering commands, and provides pixel output to display device118for display. The APD116, in at least some embodiments, includes one or more parallel processing units that perform computations in accordance with a single-instruction-multiple-data (SIMD) paradigm.

The processing system100also includes a waveguide modeler120. In at least some embodiments, the waveguide modeler120is implemented separate from or as part of one or more processors102(e.g., CPU, GPU, a combination thereof, or the like), one or more application-specific integrated circuits/circuitry (ASICs), one or more programmable logic devices, one or more other components of the processing system100, or a combination thereof. In other embodiments, the waveguide modeler120is implemented as software executable on one or more processors102. In at least some embodiments, the processor102configures the waveguide modeler120to perform one or more techniques described herein for simulating the optical performance of a diffractive waveguide.

In at least some embodiments, the waveguide modeler120is configured to model diffractive waveguides. A diffractive waveguide is a waveguide that implements diffraction gratings. A diffraction grating is an optical element having a periodic structure that diffracts light into several beams traveling in different directions (i.e., different diffraction angles). Stated differently, a diffraction grating separates (disperses) light into its constituent wavelengths (colors) such that each wavelength is diffracted at a slightly different angle. The directions or diffraction angles of the beams depend on the wave (light) incident angle to the diffraction grating, the spacing or distance between adjacent diffracting elements (e.g., grooves, slits, slots, etc.) on the diffraction grating, and the wavelength of the incident light. A diffraction grating is typically either a reflection grating that diffracts light back into the plane of incidence or a transmission grating that transmits dispersed light through the grating.

Diffraction gratings are typically implemented as one or more of input couplers (ICs), exit pupil expanders (EPE), or outcouplers (OC) of a waveguide. For example,FIG.2illustrates an example configuration of a waveguide200. In this example, the waveguide employs a first diffraction grating as an IC202to receive display light, a second diffraction grating as an EPE204to increase the size of the display exit pupil, and third diffraction grating as an OC206to direct the resulting display light toward a user's eye. If the pupil of the eye is aligned with one or more exit pupils provided by the OC206, at least a portion of the light exiting through the OC206will enter the pupil of the eye, thereby enabling the user to see a virtual image.

Referring back toFIG.1, the waveguide modeler120is configured to take parameters of a diffractive waveguide200as input and, based on this input, simulate the optical performance of the diffractive waveguide200. Examples of parameters taken as input by the waveguide modeler120include geometric properties, material properties, wavelength(s) of light to be propagated through the waveguide200, boundary conditions, mode or initial field configuration for simulating the propagation of light through the waveguide200, grating (e.g., IC202, EPE204, and OC206) parameters, grating material properties, and the like. The geometric properties include, for example, the shape and size of the waveguide200, such as length, width, and height. The material properties include, for example, the refractive indices of the materials used in the waveguide, as well as any wavelength dependence of the refractive index (dispersion). Boundary conditions specify how the field behaves at the boundaries of the waveguide200. Grating parameters include, for example, the period of the grating (the distance between similar points in the structure), the shape of the grating (e.g., square, sinusoidal, sawtooth), the duty cycle (the ratio of the width of the “teeth” to the period), and the depth or thickness of the grating. Grating material properties include, for example, the refractive indices of the materials used in the grating.

As described in greater detail below, the waveguide modeler120efficiently and accurately simulates the optical performance of a diffractive waveguide200by integrating both diffractive grating modeling to model the wave optics effect and ray tracing to model the refractive optics effect. For example, given an input light ray having a specified wavelength(s) and projected on the IC202of the waveguide200at a specified field angle, the waveguide modeler120performs diffractive grating modeling for each grating structure (e.g., IC202, an EPE204, and OC206) to generate a set of transfer matrices122. The set of transfer matrices122relates the electric field of one bounce a light ray makes on the waveguide200or grating surface to the electric field of neighboring bounces. The waveguide modeler120performs the diffractive grating modeling process for a plurality of input light rays each having a different wavelength(s), and for each input light ray projected at a plurality of different field angles. The waveguide modeler120also performs a ray tracing process for the input light ray and uses the set of transfer matrices to output a set of E-fields124for each light ray outcoupled by the OC of the waveguide. The waveguide modeler120then generates a pupil efficiency uniformity map126based on the set of output E-fields124generated during the ray tracing process for each of the plurality of input light rays field angles. The pupil efficiency uniformity map126provides a quantification of the waveguide's efficiency. For example, the pupil efficiency uniformity map126indicates, for different sampled pupil positions and field angles, the brightness of the outcoupled light rays, the color and brightness uniformity of the outcoupled light rays, and the like. In at least some embodiments, the waveguide modeler120generates a graphical representation of the pupil efficiency uniformity map126to visualize the efficiency of the waveguide200.

FIG.3shows a more detailed view of the waveguide modeler120. In at least some embodiments, the waveguide modeler120comprises a light ray simulator302, a diffraction modeler304, a ray tracer306, and a uniformity map generator308. As described in greater detail below, the light ray simulator302generates simulated light rays310(also referred to herein as “input light rays210”) for simulating the optical performance of a waveguide. In at least some embodiments, two or more of these components are combined into a single component of the waveguide modeler120.

The light ray simulator302simulates a plurality of input light rays310, each having a different wavelength or range of wavelengths. For example, the light ray simulator302simulates a first input light ray310having a red light wavelength(s), a second input light ray310having a blue light wavelength(s), and a third input light ray310having a green light wavelength(s). The light ray simulator302also simulates each of the input light rays310at different field angles of interest, which are the angles at which an input light ray is incident on the IC202(or coupled into the IC) of the waveguide200being modeled. For example, if a range of field angles to be considered during the waveguide modeling is 0 to 50 degrees, the light ray simulator302, in at least some embodiments, simulates the input ray310at each of the field angles (or a subset thereof). In other words, the light ray simulator302generates a first input light ray310having a first wavelength and projects this light ray310onto the IC202of the waveguide200being modeled at a first field angle. The light ray simulator302then simulates another instance of the first input light ray310as being projected at another field angle. This process is repeated until the first input light ray has been simulated as being projected onto the IC202for each of the field angles of interest (or a subset thereof). A similar process is also performed for each input light ray310having a different wavelength(s). In at least some embodiments, the light ray simulator302also simulates multiple instances of an input light ray310with different intensities.

The diffraction modeler304executes one or more computational techniques, such as RCWA, on each grating structure of the waveguide200being modeled and models the diffraction of the grating structures. In at least some embodiments, the diffraction modeler304takes, as input, the characteristics of the input light ray310(i.e., the incident light) and the grating structure that the light interacts with. These parameters define the optical problem to be solved. The incident light input includes, for example, the wavelength(s) of the light, the angle(s) of incidence (the direction from which the light is coming), the polarization state(s) (either transverse electric (TE), where the electric field is perpendicular to the plane of incidence, or transverse magnetic (TM, where the magnetic field is perpendicular to the plane of incidence), and the intensity or amplitude of the light. In at least some embodiments, the light ray simulator302generates one or more simulated light rays310that are used by the diffraction modeler304, or the diffraction modeler304obtains the characteristics of the incident light rays from the light ray simulator302. The grating structure input includes, for example, the geometrical parameters (e.g., shape, size, and periodicity of the features), the refractive index of the materials in the grating (which might be a function of position within the grating, if there are multiple materials), and the number of periods of the grating to be considered. In at least some embodiments, the grating is one-dimensional (varying in one direction) or two-dimensional (varying in two directions), and has any number of layers.

The diffraction modeler304takes the input and calculates the diffracted fields (reflected and transmitted) of the grating structure being modeled. For example, the diffraction modeler304defines the grating structure and incident wave by specifying the geometry, material properties, and periodicity of the grating structure, as well as the properties of the incident wave, such as its wavelength and angle of incidence. The diffraction modeler304then discretizes the grating structure by dividing the structure into layers along the direction of propagation. Each layer can have a different permittivity and permeability, but within each layer, these properties are assumed to be constant. The diffraction modeler304expands the permittivity distribution and electromagnetic fields (E-fields) within each layer into Fourier series, due to the periodicity of the grating structure. The diffraction modeler304determines boundary conditions for the E-fields at the interfaces between layers. These conditions are derived from Maxwell's equations and ensure that the E-fields are continuous across the interfaces. The diffraction modeler304calculates the E-field within each layer using the Fourier expansions and the boundary conditions. In at least some embodiments, this is performed by setting up and solving a system of linear equations, typically using matrix methods. For example, a matrix is created for the E-fields at the top and bottom of each layer. The matrix includes the coefficients of the Fourier expansions of the E-fields. Then, for each layer, the diffraction modeler304forms and solves an eigenvalue problem from the matrix formulation. This process yields the propagation constants (eigenvalues) and modal profiles (eigenvectors) of the modes within the layer. The diffraction modeler304calculates the transfer matrix of each layer, which propagates the E-fields from the bottom to the top of the layer, using the eigenvalues and eigenvectors. Then, the diffraction modeler304calculates the transfer matrix122of the whole grating structure by multiplying the layer matrices together. In at least some embodiments, the transfer matrix122of the whole grating structure is output by the diffraction modeling process and used as an input to the ray tracing process described below. From the transfer matrix of the whole structure, the diffraction modeler304calculates the E-fields that are reflected from and transmitted through the grating structure by applying the transfer matrix to the incident field. In at least some embodiments, the reflected and transmitted E-fields are also output by the diffraction modeling process, which can be used to calculate quantities of interest, such as diffraction efficiencies, reflectance, transmittance, and so on of the grating structure. In other embodiments, the diffraction modeler304stops the diffraction modeling process after the transfer matrix122of the whole grating structure is generated.

The diffraction modeling process described above is iteratively performed for each grating structure of the waveguide based on each input light ray310simulated by the light ray simulator302and each instance of the input light rays310projected at different field angles. For example, consider the waveguide200ofFIG.2and a first input light ray310having a first wavelength(s), a second input light ray310having a second wavelength(s), and a third input light ray310having a third wavelength(s). In at least some embodiments, the diffraction modeler304models the IC202by performing a plurality of iterations of the modeling process for each of the first input light ray310, the second input light ray310, and the third input light ray310. Each iteration of the plurality of iterations models the diffraction of the IC202given the same input light ray310but with a different field/incident angle. For example, the diffraction modeler304performs a first iteration of the diffraction modeling process for the IC202based on the first input light ray310projected at a first field angle and generates a first transfer matrix122for the IC202based thereon. The diffraction modeler304then performs additional iterations of the diffraction modeling process for the IC202, with each additional iteration projecting the first input light ray310at a different field angle, and generates additional transfer matrices122for the IC202based thereon. Additional iterations are then performed for each of the second input light ray310and the third input light ray310. The diffraction modeler304repeats this process for each of the EPE204and the OC206. As such, the diffraction modeler304generates a first plurality of transfer matrices122-1for the IC202, a second plurality of transfer matrices122-2for the EPE204, and a third plurality of transfer matrices122-3for the OC206. In at least some embodiments, the diffraction modeler304also generates one or more transfer matrices122-4for the internal world side-facing area of the waveguide200without a grating surface (also referred to herein as “world side background area”) and one or more transfer matrices122-5for the internal eye side facing area of the waveguide200without a grating surface (also referred to herein as “eye side background area”).

The ray tracer306performs a ray tracing process for each input light ray310to simulate the propagation of the input light ray310through the waveguide200being modeled. In at least some embodiments, the ray tracing process is an iterative process that is performed for each input light ray310, each instance of the input light rays310projected at different field angles, and each incident ray position of a plurality of incident ray positions (e.g., spatial positions on the IC202at which the input light rays310first contact the IC202). In at least some embodiments, the ray tracing process performed by the ray tracer306is based on deterministic ray tracing and takes into consideration the unique physics of the waveguide200, i.e., coherent combination of rays during propagations, to continually degenerate rays during the ray tracing. The concept of a “node” is used to determine the coherence state of the ray. For light sources, such as light-emitting diodes (LEDs), rays arriving at the same node have identical optical path lengths (OPLs) and are summed coherently, whereas rays at different nodes have different OPLs and are summed incoherent. Also, in at least some embodiments, one or more diffraction orders that have a low diffraction efficiency are ignored during the ray tracing process. For example, the 0thorder reflection on the world side background area of the waveguide200having an anti-reflective coating, the +/−2ndorder of the IC202, and the +2ndorder of the OC206can also be disregarded. However, in other embodiments, one or more of these orders are considered during the ray tracing process.

In at least some embodiments, the input taken by ray tracer306includes, for example, one or more of the initial conditions, wave structure information, material properties, boundary conditions, and the like. The initial conditions include the incident position (starting position) and field angle (direction) of the input light ray310being traced. The initial conditions, in at least some embodiments, also include the wavelength and polarization of the input light ray310. The waveguide structure information includes information regarding the structure and geometry of the waveguide200being modeled, such as the shape, size, and thickness of the waveguide. In at least some embodiments, the waveguide structure information also includes information regarding any variations in the waveguide's structure, such as bends, tapers, or irregularities. The material properties include information about the optical properties of the materials that make up the waveguide200being modeled, such as the refractive index of each material. In at least some embodiments, the material properties also include information regarding any variations in the material properties, such as gradations in the refractive index. The boundary conditions indicate the behavior of light rays at the boundaries of the waveguide200. For example, the boundary conditions include information about how light is reflected or transmitted at the boundaries and special conditions, such as periodic or absorbing boundaries.

In at least some embodiments, the input light ray310is represented as ray (i,i), whereiis the incident direction vector (also referred to here as “k-vector” or “wave vector”) andiis the incident coordinate on the IC202. The k-vector of a light ray corresponds to or illustrates a direction in which the light ray propagates through a waveguide. Given the input light ray310, ray (i,i), the ray tracer306generates a grid of nodes where the input light ray310intersects the waveguide200and grating surfaces (e.g., IC202, EPE204, and OC206). Stated differently, each bounce the input light ray310makes on the waveguide200or grating surface is a node in the grid of nodes.FIG.4shows one example of a grid400of nodes402for the waveguide200ofFIG.2representing where an input light ray intersects the waveguide200and surfaces of the IC202, EPE204, and OC206.

In at least some embodiments, the ray tracer306determines the positions of the nodes402in the grid400using a k-space diagram generated for the waveguide.FIG.5shows one example of a k-space diagram500representing display light propagating through a waveguide, such as the waveguide200ofFIG.2. A k-space diagram is a tool used in optical design to represent directions of light rays that propagate within a waveguide. Stated differently, a k-space diagram shows the angles at which light is coupled into a waveguide. In the k-space diagram500, an inner refractive boundary502is depicted as a circle with radius of n=1, the refractive index associated with the external transmission medium (air). An outer refractive boundary504corresponds to an effective refractive index of the medium of the waveguide200. In the context of the k-space diagram500, for red, green, blue (RGB) display light to be successfully and accurately directed to an eye of a user via the waveguide200with the indicated refractive index, each red, green, and blue component of that display light enters the waveguide system from an external position506, which is included in the space depicted within inner refractive boundary502. The color components are directed along one or more paths within the waveguide200via total internal reflection (TIR) (light that undergoes TIR within the waveguide200resides in the space depicted between inner refractive boundary502and outer refractive boundary504) and are then redirected to exit the waveguide200(and thereby return to the external space within inner refractive boundary502within which light does not undergo TIR). Display light components represented between the inner refractive boundary502and outer refractive boundary504are propagated to the user via the waveguide200. Any display light components represented outside the outer refractive boundary504(of which there are none in the k-space representation500) are non-propagating and cannot exist.

Initially, display light entering the waveguide200at the incoupler forms an image that is centered at or around the origin of the k-space representation500. The image is initially disposed at a first position506with respect to k-space. Upon redirection of the display light by the IC202, the image is shifted in k-space to a second position508, corresponding to a shift in the negative kyand kxdimensions. Upon redirection of the display light by the EPE204, the image is shifted in k-space to a third position510, corresponding to a shift in the positive kydimension and the negative kxdimension. Upon redirection of the display light by the OC206, the image is shifted in k-space back to the first position506, corresponding to a shift in the positive kxdimension. In the present example, it is assumed that the angle at which the display light enters the waveguide system via the IC202is the same as or substantially the same as (e.g., within 5% of) the angle at which the display light exits the waveguide200via the OC206.

In at least some embodiments, the ray tracer306obtains the angles at which light propagates through the waveguide based on the k-space diagram500and uses these propagation angles to determine the position in x-space (real space) at which a light ray hits the surface of the waveguide structure and surfaces of the IC202, EPE204, and OC206. The ray tracer306, in at least some embodiments, calculates the propagation angle(s) of an input light ray310based on the k-vector(s) of the ray310obtained from the k-space diagram500. For example, consider a k-vector obtained for an input light ray310at the projector (light source) output represented as:

k⇀1=[k1⁢x,k1⁢x,k1⁢z]=n⁢k0,with⁢k1⁢z<0,(EQ.1)where k0is the k-vector in vacuum and n is the refractive index of the material through which the light ray is traveling.

The k-vector1is described in terms of its Cartesian components (kx, ky, kz) representing the light ray's spatial frequencies along the x, y, and z axes. The ray tracer306normalizes the k-vector1to the k-vector k0in a vacuum as follows:

The ray tracer306then obtains the direction cosinec1of the light ray from the components ({circumflex over (k)}1x, {circumflex over (k)}1y, {circumflex over (k)}1z) of the normalized vector {circumflex over (k)}1as follows:

The ray tracer306repeats the above process for the k-vector2incident on the EPE204of the waveguide200and the k-vector3incident on the OC206of the waveguide200to obtain the normalized vectors {circumflex over (k)}2and {circumflex over (k)}3and corresponding directional cosinesc2andc3, respectively, as follows:

d⇀c⁢2=k⇀2nE⁢P⁢E⁢with⁢d⇀c⁢2[2]>0,(EQ.9)where nEPEis the refractive index of the EPE204material,

d⇀c⁢3=k⇀3nO⁢C⁢with⁢d⇀c⁢3[2]>0,(EQ.10)where nOCis the refractive index of the OC material,c2·r representsc2reflected by the x-y plane, andc3·r representsc3reflected by the x-y plane.

The notationc2[2] in EQ. 9 andc3[2] in EQ. 10 refers to the third element of thec2andc3, respectively, such thatc2[2]=cos c2andc3[2]=cos c3. Therefore,c2[2]>0 andc3[2]>0 indicate that the k-vectors2and3are pointing towards the z>0 direction (propagating upwards). Otherwise, the k-vectors2and3are pointing towards the z<0 direction (propagating downwards).

The ray tracer306calculates the propagation angle(s) for the input light ray310based on the directional cosines determined for the light ray310. For example, the ray tracer306calculates the propagation angle θ2of the light ray310at the EPE204and the propagation angle θ3of the light ray310at the OC206using the inverse cosine function (a cos) as follows:

θ2=a⁢cos⁢d⇀c⁢2[2],(EQ.11)andθ3=a⁢cos⁢d⇀c⁢3[2].(EQ.12)The ray tracer306, in at least some embodiments, does not calculate the propagation angle(s) for the input light ray310at the IC202because the direction and propagation of the light ray310entering the waveguide through the IC202are already known or controlled.

Using the propagation angles, θ2and θ3, and directional cosines,c2andc3, the ray tracer306determines the position of each bounce and the spacing between each bounce that the light ray310makes on the waveguide200, IC202, EP204, and OC206using, for example, geometric calculations and the law of reflection. The ray tracer306generates nodes402in the grid400based on the determined bounce positions and spacings. Stated differently, node402in the gride of nodes402represents the x-y position, as calculated based on the k-space diagram500, at which a light ray hits the surface of the waveguide structure or surfaces of the IC202, EPE204, and OC206. For example, referring toFIG.4, the nodes402are spaced according to a first bounce spacing, rD, in the up-down direction and a second bounce spacing, rL, in the right-to-left direction. The bounce spacings rDand rL, which represent the length of a bounce, are calculated by the ray tracer306as follows:

rD=2⁢to⁢tan⁢θ2,(EQ.13)where tois the thickness of the waveguide, and

The ray tracer306also calculates a bounce spacing vector, which represents bounce direction, for each of the bounce spacings rDand rLas follows:

r⇀D=rD[cos⁢ℓ2,sin⁢ℓ2],(EQ.15)where2is the azimuth angle ofc2, and

r⇀L=rL[cos⁢ℓ3,sin⁢ℓ3],(EQ.16)where3is the azimuth angle ofc3.

The azimuth angles2and3of the directional cosinesc2andc3are the angles in the xy-plane of the waveguide200from the positive x-axis towards the positive y-axis. Stated differently, the azimuth angles2and3give the direction of the light ray310in the xy-plane of the waveguide200. The ray tracer306determines the azimuth angles2and3using an inverse tangent or arctangent function as follows:

The coordinate [i,j] of a node402in the grid400is denoted as a matrix grid having the shape n×m×2 such that:

grid[i,j]=[xi,yj]=grid[0,0]+i·r⇀D+j·r⇀L(EQ.19)where grid [0,0] is the incident coordinate of the input light ray on the IC.As such, starting at the incident node position (e.g., grid coordinate [0, 0]) of the input light ray310on the IC202, the ray tracer306determines the position of each node402in the grid400according to EQ19. For example, the ray tracer306calculates each [xi,yi] position of a node402in the indexed [i,j] grid400starting from position [0, 0] plus i shifts in the downward direction and j shifts in the leftward direction (grid [0,0]+i·D+j·L). In at least some embodiments, the ray tracer306calculates a grid400of nodes402for each input light ray310simulated with a different field angle.

After the ray tracer306generates the grid400of nodes402for an input light ray310having a specified field angle, the ray tracer306recursively calculates the E-fields of each node402using the transfer matrices122generated during the diffraction modeling process. The transfer matrices122relate the E-field of one node402to its neighboring node(s)402. As such, the ray tracer306determines the E-field of each node402by recursively performing matrix multiplication using one or more transfer matrices122generated for the IC202, EPE204, and OC206of the waveguide200and neighboring E-fields.

In the following description, transfer matrix T is denoted as T(grating_name, incident_direction, order_number), which is the 3×3 transfer matrix for grating_name at incident_direction with order_number. The incident E-fieldincis represented as the following 3×1 complex vector:

E⇀i⁢n⁢c=(Ei⁢x,Ei⁢y,Ei⁢z)T,(EQ.20)which follows either S polarization such thatinc=(1,0,0)T, or P polarizationinc=(0,1,0)T.

The E-field propagating downward (along the 0thorder) at node [i,j] is represented as:

Within the IC202area of the grid400, the ray tracer306calculates the downward propagating E-fieldDfor the first bounce, which is a single diffraction event, at the grid position [0, 0] as follows:

E⇀D[0,0]=T⁡(IC,d⇀c⁢1,1)⁢E⇀i⁢n⁢c.(EQ.22)As such, the downward propagating E-fieldDfor the first bounce at the IC202is calculated by multiplying IC transfer matrix122-1(with directionc1along the +1 order) by the incident E-fieldinc.

The ray tracer306then moves to the next node402in the grid400. If the position, grid [i,j], of this node402is within the IC202where multiple bounces occur, the ray tracer306only tracks the 0thorder reflection and calculates the downward propagating E-fieldpfor the node402as follows:

E⇀D[i,0]=T⁡(IC,d⇀c⁢2,0)⁢T⁡(W⁢S,d⇀c⁢2.r,0)⁢E⇀D[i-1,0]⁢exp⁢(i⁢Φ)(EQ.23)where WS is the world side background area of the waveguide, i in exp (iΦ) is the imaginary unit, and Φ is the propagation phase along the 0thorder direction from node [i−1, 0] to node [i, 0], where:

Φ=k⇀·r⇀=k0(kx⁢rx+ky⁢ry+2⁢kz⁢t⁡(x,y)),(EQ.24)and where t(x,y) is the thickness of the waveguide at position (x,y) and

are the reducedvector of the light ray.

As such, the E-fieldDfor a node402within the area of IC202in the grid400is calculated by multiplying together the IC transfer matrix122-1(with directionc2along the 0thorder), the WS transfer matrix122-4(with reflected directionc2along the 0thorder), the E-fieldDof the neighboring node402at grid position [i−1, 0], and the exponential function of the propagation phase.

If the grid position [i,j] of the node402is outside of the IC202, the ray tracer306only tracks 0thorder reflection and calculates the downward propagating E-fieldDfor the node402as follows:

E⇀D[i,0]=T⁡(E⁢S,d⇀c⁢2,0)⁢T⁡(W⁢S,d⇀c⁢2·r,0)⁢E⇀D[i-1,0]⁢exp⁡(i⁢Φ)(EQ.25)where ES is the eye side background area of the waveguide.

As such, the E-fieldDfor a node402outside of the IC202area in the grid400is calculated by multiplying together the ES transfer matrix122-5(with directionc2along the 0thorder), the WS transfer matrix122-4(with reflected directionc2along the 0thorder), the E-fieldDof the neighboring node402at grid position [i−1, 0], and the exponential function of the propagation phase.

The ray tracer306continues stepping through each node402in the first column (j=0) and performs the E-field calculation process described above until the next node402is within the EPE204area of the waveguide200. In the EPE204, the E-fields are split into two directions at each node [ij]. For example,FIG.6shows a portion of the nodes402in the grid400ofFIG.4that is within the EPE204area. In this example, the right side ofFIG.6shows that the E-field of each node402within the EPE204area is split into two E-fields, a first E-field,D, that travels up-to-down (along the 0thorder) with directionc2and a second E-field,L, that travels left-to-right (along the +1 order) with directionc3. The left side ofFIG.6further shows that each node402within the EPE204area will have two input E-fields,DandL, and two output E-fields,DandL. For example, node [i,j] inFIG.6receives the E-fieldD[i−1,j] from node [i−1,j] and the E-fieldL[i,j−1] from node [i,j−1], and outputs two E-fields,D[i,j] andL[i,j]. The interaction at node [i,j] between the two input E-fields is referred to as coherent field summation.

For the first column (j=0) in the grid400and starting from i=a+1, every node402in the first column that is within the EPE204area is only dependent on the node402above it. For example, inFIG.6, the right-most column is the first column (j=0). The E-field for each node402in this column of the grid400that is within the EPE204area is only dependent on the node above it, as there are no nodes402to the right of this column that are within the EPE204area. Therefore, starting at grid position [a+1, 0], the ray tracer306determines the E-fields,DandL, for each node402within the EPE204area in the first column (j=0) as follows:

E⇀D[i,0]=T⁡(E⁢P⁢E,d⇀c⁢2,0)⁢T⁡(W⁢S,d⇀c⁢2·r,0)⁢E⇀D[i-1,0]⁢exp⁡(i⁢ΦD),(EQ.26)E⇀L[i,0]=T⁡(E⁢P⁢E,d⇀c⁢2,1)⁢T⁡(W⁢S,d⇀c⁢2·r,0)⁢E⇀D[i-1,0]⁢exp⁡(i⁢ΦL).(EQ.27)where ΦDis the downward direction propagation phase of the input light ray310and ΦLis the leftward direction propagation phase of the input light ray310.

As such, the E-fieldDfor a node402in the first column of the grid400within the EPE204area is calculated by multiplying together the EPE transfer matrix122-2(with directionc2along the 0thorder), the WS transfer matrix122-4(with reflected directionc2along the 0thorder), the E-fieldDof the neighboring node402at grid position [i−1, 0], and the exponential function of the propagation phase along the 0thorder. Similarly, the leftward propagating E-fieldLfor the node402is calculated by multiplying together the EPE transfer matrix122-2(with directionc2along the +1 order), the WS transfer matrix122-4(with reflected directionc2along the 0thorder), the E-fieldDof the neighboring node402at grid position [i−1, 0], and the exponential function of the propagation phase along the +1 order.

For each remaining column in the grid400, every node402that is within the EPE204area is dependent on two neighboring nodes402; that is the node402directly above the current node402and the node402directly to the right of the current node402. For example, inFIG.6, the E-fields of node [i,j] are dependent on the E-field of node [i−1,j] and the E-field from node [i,j−1]. Therefore, for each subsequent column (j=0+b, where b>0) after the first column (j=0), the ray tracer306determines the E-fields,DandL, for each node402within the EPE204area as follows:

As such, the E-fieldDfor a node402in a subsequent column of the grid400within the EPE204area is calculated by summing the product of multiplying together the EPE transfer matrix122-2(with directionc2along the 0thorder), the WS transfer matrix122-4(with reflected directionc2along the 0thorder), the E-fieldDof the neighboring node402at grid position [i−1,j], and the exponential function of the propagation phase along the 0thorder with the product of multiplying together the EPE transfer matrix122-2(with directionc3along the −1 order), the WS transfer matrix122-4(with reflected directionc3along the 0thorder), the E-fieldLof the neighboring node402at grid position [i,j−1], and the exponential function of the propagation phase along the +1 order. Similarly, the leftward propagating E-fieldLfor the node402is calculated by summing the product of multiplying together the EPE transfer matrix122-2(with directionc2along the +1 order), the WS transfer matrix122-4(with reflected directionc2along the 0thorder), the E-fieldDof the neighboring node402at grid position [i−1,j], and the exponential function of the propagation phase along the 0thorder with the product of multiplying together the EPE transfer matrix122-2(with directionc3along the 0thorder), the WS transfer matrix122-4(with reflected directionc3along the 0thorder), the E-fieldLof the neighboring node402at grid position [i,j−1], and the exponential function of the propagation phase along the +1 order.

When a node402in the grid400is outside of the EPE204area (and OC206area), such as node [7,0] or [7,1] inFIG.4, total internal reflection (TIR) occurs. Therefore, for these nodes402, the ray tracer306calculates the E-fields as follows:

The ray tracer306continues stepping through each node402in each subsequent column and performs the E-field calculation process described above until the next node402is within the OC206area of the waveguide200. In the OC206, the E-fields can propagate in three directions:0thorder-from left to right, denoted byL[i,j],+1 order Transmissive (T)—outcouple to eye side, denoted byout[i,j], and+1 order Reflective (R)—outcouple to world side, denoted byworld[i,j].

In at least some embodiments, the E-fieldworld[i,j] is not considered by the ray tracer306. In these embodiments, the ray tracer306calculates the output E-fields124of the nodes402within the OC area206as follows:

As such, the E-fieldLfor a node402within the OC206area is calculated by multiplying together the OC transfer matrix122-3(with directionc3along the 0thorder), the WS transfer matrix122-4(with reflected directionc3along the 0thorder), the E-fieldLof the neighboring node402at grid position [i,j−1], and the exponential function of the propagation phase along the +1 order. Similarly, the output E-fieldoutis calculated by multiplying together the OC transfer matrix122-3(with directionc3along the +1 order), the WS transfer matrix122-4(with reflected directionc3along the 0thorder), the E-fieldDof the neighboring node402at grid position [i,j−1], and the exponential function of the propagation phase along the +1 order. In at least some embodiments, the output of the ray tracer306is theoutelectric fields124for an input light ray310have a specified wavelength, field of view, incident position on the IC202, and polarization. Theoutelectric fields124represent the near-field distribution of the replicated pupils. The uniformity of the near-field over the OC206determines the eyebox non-uniformity.

In at least some embodiments, after the ray tracer306outputs theoutelectric fields124of the outcoupling nodes402of the OC204for the current iteration, the ray tracer306repeats the ray tracing process described above for the input light ray310but with a different polarization. For example, after the ray tracer306determines theoutelectric fields124for an input ray310having Wavelength_1, FOV_1, IncidentPos_1, and Polarization_S, the ray tracer306repeats the raytracing process for this input ray310but for Polariation_P. Therefore, in at least some embodiments, the ray tracer306outputs two sets ofoutelectric fields124for an instance of input light ray310(e.g., an input light ray having a specified wavelength, field angle, and incident position) comprised of a first set sets ofoutelectric fields124based on S polarization and a second set of sets ofoutelectric fields124based on P polarization. If the display being modeled is unpolarized, the ray tracer306sums the two sets ofelectric fields124. Theelectric fields124provide information such as the wavelength, the propagation direction, intensity, and polarization of the outcoupled ray. Stated differently, theelectric fields124gathers all the information associated with the light that is directed into the eyebox (the box where the user's pupil is located). Theelectric fields124can be used to render the displayed image, predict the distribution of color and brightness over the FOV or across different locations of the eyebox, and the like.

The ray tracer306performs additional iterations of the ray tracing process for the current input light ray310to outputoutelectric fields124based on each remaining incident ray position of the plurality of incident ray positions. Then, the ray tracer306repeats the ray tracing process for the current input light ray310but for a different field angle and each incident ray position of the plurality of incident ray positions. After the ray tracer306completes the ray tracing process for the current input light ray310and each field angle, the ray tracer306performs the iterative ray tracing process for a new input light ray310having a different wavelength until the ray tracer306has generated theoutelectric fields124for all input light rays310, including theoutelectric fields124for each instance of the input light rays310projected at different field angles and at each incident ray position of the plurality of incident ray positions.

It should be understood that although the ray tracing process was described above with respect to one-dimensional (1D) grating structures, the ray tracing process is also applicable to two-dimensional (2D) grating structures. For a 2D grating structure, the ray tracer306considers six diffraction orders (00, 10, 01, 22, 21, and 12) between the inner refractive boundary and the outer refractive boundary of the corresponding k-space diagram, where every two orders are connected by a transfer matrix. The nodes402in a grid400generated for a waveguide comprising a 2D grating structure form a six-dimensional matrix denoted by index [i, j, k, o, p, q]. As such, instead of the E-fields124of a node402in the grid400being dependent upon two neighboring nodes, as for the 1D grating, the E-fields124are now dependent upon six neighboring nodes402.

The ray tracer306determines the node positions in the grid400from a K-space diagram according to:

r[i,j,k,o,p,q]=i·r0⁢0+j·r1⁢0+k·r0⁢1+o·r2⁢2+P·r2⁢1+⁢1·r1⁢2,(EQ.36)where rmnis the bounce vector along the mn order. Also, instead of two propagation directions,0(e.g.,D) and1(e.g.,L) at each node in a 1D grating, a 2D grating has seven different propagation directions on each node denoted by the following E-field vectors:00,10,01,11,21,12,22. The ray tracer306performs the ray tracing process described above according to the following near-field recursive update rule to obtain theoutelectric fields124at the OC206:

In at least some embodiments, the map generator308receives theoutelectric fields124generated by the ray tracer306and generates a pupil efficiency uniformity map126based on theoutelectric fields124. The pupil efficiency uniformity map126provides a quantification of the waveguide's efficiency. For example, the pupil efficiency uniformity map126indicates, for different sampled pupil positions and field angles, the brightness of the outcoupled light rays, the color and brightness uniformity of the outcoupled light rays, and the like.

FIG.7shows one example of a pupil efficiency uniformity map726generated by the map generator308based on theoutelectric fields124output by the ray tracing process described above. In this example, a graphical representation of a pupil efficiency uniformity map726is generated by the map generator308to visualize the efficiency of the waveguide200. Each block702within the map700represents an image as seen by the pupil at a different sampled pupil position. For example, block702-1represents an image generated by the map generator308based on theoutelectric fields124for pupil position AA, whereas block702-2represent an image generated by the map generator308based on theoutelectric fields124for pupil position EE. Each block702is associated with an output power P, which corresponds to the efficiency of the waveguide for that pupil position. The output power P is the E-field squared and integrated over the pupil area/position. For another pupil position, there is another output power P. Therefore, as shown in theFIG.7, some blocks702are brighter than other blocks since the output power P is dependent upon the pupil position. The output power P is also dependent on the outcoupling angle of the light ray. As such, the same pupil position within the map726has a different efficiency/power for different outcoupling angles.

In at least some embodiments, the map generator308calculates the output power P for each pupil position of a pupil efficiency uniformity map126by integrating the Poynting vector inside a pupil projected onto the waveguide surface as follows:

Pout=∫∫pupilSz⁢dxdy=∫∫pupil❘"\[LeftBracketingBar]"E⇀out❘"\[RightBracketingBar]"2⁢dxdy=∑i,j∈pupil❘"\[LeftBracketingBar]"E⇀out[i,j]❘"\[RightBracketingBar]"2,(EQ.38)where Szis the Poynting vector along the z direction. In at least some embodiments, the integral is over intensity instead of the E-field because, for some light sources, such as light emitting diodes (LEDs), different points on the exit pupil of the light engine are incoherent and different nodes on the OC are incoherent. The waveguide modeler120, in at least some embodiments, calculates the efficiency of the waveguide for a given pupil position as:

Pin=∫∫EP❘"\[LeftBracketingBar]"E⇀inc❘"\[RightBracketingBar]"2⁢dxdy.(EQ.40)For unpolarized performance, the waveguide modeler120takes the average of efficiencies generated by S and P polarized incident field.

In at least some embodiments, the pupil efficiency uniformity map126is used to adjust the design parameters of the waveguide200and its components, such as the grating structures, to increase the performance of the waveguide200. For example,FIG.8shows another example of a pupil efficiency uniformity map826. The map826is a chromaticity map of a waveguide having poor color uniformity calculated based on theoutelectric fields124output by the ray tracing process described above. In this example, the target waveguide display is gray but the waveguide outputs blueish light (represented by the lighter shading) in the right portion801of the eyebox and orangish light (represented by the darker shading) in the left portion803eyebox together with non-uniformity across the FOV in the same eyebox location. As such, based on the color non-uniformity determined from the map826, one or more waveguide design parameters can be adjusted to improve, the color uniformity over the eyebox and FOV, as shown in the pupil efficiency uniformity map926ofFIG.9.

FIG.10shows another example of a pupil efficiency uniformity map1026. In this example, the map1026is a luminance map of a waveguide having poor brightness uniformity calculated based on theoutelectric fields124output by the ray tracing process described above. The map1026shows that the waveguide display is brighter towards the temple (upper left) direction and dimmer towards the nasal (lower right) direction. There is also brightness non-uniformity across the FOV in the same eyebox location. As such, based on the brightness non-uniformity determined from the map1026, one or more waveguide design parameters can be adjusted to improve brightness uniformity over the eyebox and FOV, as shown in the pupil efficiency uniformity map1126ofFIG.10.

In at least some embodiments, the ray tracer306is configured to represent light ray bounces using mechanisms other than the grid400ofFIG.4.FIG.12shows one example of these additional mechanisms. InFIG.12, light ray bounces for a waveguide implementing 2D grating structures are represented using a tree configuration. For example,FIG.12shows a tree1200generated for light bounces having an incident (root) node1202. Every bounce branches one node1202of the tree1200into six children nodes1202. For example,FIG.12shows that the incident node1202-1is branched into six child nodes1202-2to1202-7. The number of child nodes is determined by the number of diffraction orders the grating at parent node position (x,y) has. Each of the nodes1202in the tree1200is associated with a node coordinate (e.g., [0,0,0,0,0,0] and each branch is associated with a diffraction order (e.g., [1,0]). Each child node1202generated by a bounce has eight outgoing orders including six guided orders and two out-coupled orders. Therefore, one light ray is uniquely represented by a row vector [x, y, i, j. k. o, p, q, u, v,x,y,z], where (x,y) is the location of the node1202, (i,j,k,o,p,q) is the counter of bounces in each direction in x space, (u,v) is the diffraction order in k-space, and (x,y,z) is the complex E-field vector. As shown inFIG.13, in at least some embodiments, the (i,j,k,o,p,q) portion of a node's row vector is replaced by the optical path length d as follows:

[x,y,i,j·k·o,p,q,u,v,⁠E⇀x,E⇀y,E⇀z]⁠=⁠[x,y,d,u,v,E⇀x,E⇀y,E⇀z],(EQ.41)and the condition for coherent addition becomes Δd<coherence length.

The ray tracer306traverses the tree1200by calculating a child node1202E-field from its parent node1202. For example, the E-field for child node1202-8is calculated as:

If (x,y) is outside of the grating area (e.g., the corresponding polygon inFIG.2), the ray tracer306does not generate the row (level) of the tree1200. In each layer of the tree1200, the ray tracer306detects the rows with the same index [i, j, k, o, p, q, u, v] or [d, u, v]. For example, child nodes1202-8and1202-9have the same index, child nodes1202-10and1202-11have the same index, and child nodes1201-12and1202-13have the same index. The ray tracer306combines the child nodes (rows) having the same index by adding the last three columns [x,y,z] of the nodes, which reduces the number of rows before expanding the next layer of the tree1200. In at least some embodiments, if ray tracer306determines that the power of a light ray is less than a threshold (e.g., ||2<ε, the ray tracer306deletes the row. In other words, the ray tracer306stops tracing a light ray if its power is less than a specified threshold ε.

FIG.14andFIG.15together illustrate an example method1400for performing one or more of the techniques described herein to efficiently and accurately simulate the optical performance of a diffractive waveguide. It should be understood that the processes described below with respect to method1400have been described above in greater detail with reference toFIG.1toFIG.13. The method1400is not limited to the sequence of operations shown inFIG.14andFIG.15, as at least some of the operations can be performed in parallel or in a different sequence. Moreover, in at least some implementations, the method1400can include one or more different operations than those shown inFIGS.14and15.

At block1402, the waveguide modeler120selects a characteristic, such as a wavelength or range of wavelengths, for an input light ray310. At block1404, the waveguide modeler120selects another characteristic, such as a field angle, for the input light ray310. At block1406, the waveguide modeler120executes one or more computational techniques, such as RCWA, on each grating structure of the waveguide200being modeled and models the diffraction of the grating structures based on an input light ray310having the selected wavelength and field angle. This process generates a set of transfer matrices122for each grating structure (e.g., IC202, EPE204, and OC206) of the waveguide200. At block1408, the waveguide modeler120initiates a ray tracing process and selects another characteristic, such as incident ray position, for the input light ray310. The waveguide modeler120simulates the input light ray310having the selected wavelength and field angle as being projected on the IC202of the waveguide being modeled200at the incident ray position.

At block1410, the waveguide modeler120determines the bounce positions at which the input light ray310hits waveguide structure, IC202, EPE204, and OC206. As described above, with respect toFIG.3toFIG.5, the waveguide modeler120determines the bounces of the input light ray based on the incident position and the k-space diagram500associated with the waveguide200according to EQ. 2 to EQ. 19. At block1412, the waveguide modeler120generates a grid400(or tree) of nodes402with each node402representing a bounce position. Stated differently, the position of a node402in the grid400corresponds to one of the bounce positions determined for the input light ray.

At block1414, the waveguide modeler120selects another characteristic, such as polarization, for the input light ray310. At block1416, the waveguide modeler120steps through the nodes402in the grid400and recursively determines the E-field(s) for each node402using the set of transfer matrices122generated at block1406and according to EQ. 20 to EQ. 37. At block1418, the waveguide modeler120outputs the set of E-fields124of the outcoupling nodes402calculated for the OC206of the waveguide200. This set of E-fields124is stored as part of a near-field map generated for the waveguide200being modeled. At block1420, if the waveguide modeler120determines that there is an additional polarization to be considered, the process returns to block1414, and the waveguide modeler120selects another polarization for the input light ray310. The waveguide modeler120then steps through the nodes402and recursively determines the E-field(s) for each node402based on the input light ray310having the newly selected polarization.

At block1422, if all polarizations have been considered, the waveguide modeler120determines if all incident ray positions for the input light ray310have been considered. If at least one incident ray position remains to be considered, the process returns to block1408, and the waveguide modeler120selects a new incident position for the input light ray310. The processes at block1410to block1422are then repeated for the input light ray310projected at the newly selected incident position. At block1424, if all incident ray positions have been considered, the waveguide modeler120determines if all field angles for the input light ray310have been considered. If at least one field angle remains to be considered, the process returns to block1404, and the waveguide modeler120selects a new field angle for the input light ray310. The processes at block1406to block1424are then repeated for the input light ray310simulated at the newly selected field angle. At block1426, if all field angles have been considered, the waveguide modeler120determines if all wavelengths or ranges of wavelengths have been considered. If at least one wavelength or range of wavelengths remains to be considered, the process returns to block1402, and the waveguide modeler120selects a new wavelength or range of wavelengths. The processes at block1404to block1426are then repeated using input light ray310having the newly selected wavelength or range of wavelengths. As such, multiple iterations of the diffraction process at block1406are performed for each diffractive grating of the waveguide200based on each input light ray310with a different combination of wavelength and field angle.

At block1428, if all wavelengths or ranges of wavelengths have been considered, the near-field map, which has been generated based on all of the E-fields124output at block1418for all input light ray instances (e.g., different wavelengths, field angles, and incident ray positions), is converted to a pupil efficiency uniformity map126(far-field map) according to EQ. 38 to EQ. 40. At block1430, the waveguide modeler120(or another component or system) uses the uniformity map126to determine if the uniformity of one or more attributes (e.g., color or brightness) of the waveguide200satisfies at least one uniformity threshold. At block1432, if the uniformity satisfies the at least one uniformity threshold, the process ends. At block1434, if the uniformity does not satisfy the at least one uniformity threshold, one or more design parameters of the waveguide200are adjusted and process returns to block1402.

FIG.16illustrates an example display system1600, such as a near-to-eye device or a wearable head mounted display (HMD), capable of implementing a waveguide designed based on one or more of the waveguide optical performance simulation techniques described herein. It should be noted that, although the apparatuses and techniques described herein are not limited to this particular example, but instead may be implemented in any of a variety of display systems using the guidelines provided herein. In at least some embodiments, the display system1600comprises a support structure1602that includes an arm1604, which houses an image source, such as laser projection system, configured to project images toward the eye of a user such that the user perceives the projected images as being displayed in FOV area1606of a display at one or both of lens elements1608,1610. In the depicted embodiment, the display system1600is a near-eye display system that includes the support structure1602configured to be worn on the head of a user and has a general shape and appearance of an eyeglasses frame. The support structure1602includes various components to facilitate the projection of such images toward the eye of the user, such as a laser projector, an optical scanner, and a waveguide, such as the waveguide200described above with respect toFIG.1toFIG.15. In at least some embodiments, the support structure1602further includes various sensors, such as one or more front-facing cameras, rear-facing cameras, other light sensors, motion sensors, accelerometers, and the like. The support structure1602further can include one or more radio frequency (RF) interfaces or other wireless interfaces, such as a Bluetooth™ interface, a Wireless Fidelity (WiFi) interface, and the like.

Further, in at least some embodiments, the support structure1602includes one or more batteries or other portable power sources for supplying power to the electrical components of the display system1600. In at least some embodiments, some or all of these components of the display system1600are fully or partially contained within an inner volume of support structure1602, such as within the arm1604in region1612of the support structure1602. It should be noted that while an example form factor is depicted, it will be appreciated that in other embodiments, the display system1600may have a different shape and appearance from the eyeglasses frame depicted inFIG.16.

One or both of the lens elements1608,1610are used by the display system1600to provide an augmented reality (AR) or a mixed reality (MR) display in which rendered graphical content is superimposed over or otherwise provided in conjunction with a real-world view as perceived by the user through the lens elements1608,1610. For example, laser light used to form a perceptible image or series of images may be projected by a laser projector of the display system1600onto the eye of the user via a series of optical elements, such as a waveguide (e.g., the waveguide200) formed at least partially in the corresponding lens element, one or more scan mirrors, and one or more optical relays. Thus, one or both of the lens elements1608,1610include at least a portion of a waveguide that routes display light received by an input coupler, or multiple input couplers, of the waveguide to an output coupler of the waveguide, which outputs the display light toward an eye of a user of the display system1600. The display light is modulated and scanned onto the eye of the user such that the user perceives the display light as an image. In addition, each of the lens elements1608,1610is sufficiently transparent to allow a user to see through the lens elements to provide a field of view of the user's real-world environment such that the image appears superimposed over at least a portion of the real-world environment.

In at least some embodiments, the projector is a matrix-based projector, a digital light processing-based projector, a scanning laser projector, or any combination of a modulative light source such as a laser or one or more light-emitting diodes (LEDs) and a dynamic reflector mechanism such as one or more dynamic scanners or digital light processors. The projector, in at least some embodiments, includes multiple laser diodes (e.g., a red laser diode, a green laser diode, and a blue laser diode) and at least one scan mirror (e.g., two one-dimensional scan mirrors, which may be micro-electromechanical system (MEMS)-based or piezo-based). The projector is communicatively coupled to the controller and a non-transitory processor-readable storage medium or memory storing processor-executable instructions and other data that, when executed by the controller, cause the controller to control the operation of the projector. In at least some embodiments, the controller controls a scan area size and scan area location for the projector and is communicatively coupled to a processor (not shown) that generates content to be displayed at the display system1600. The projector scans light over a variable area, designated the FOV area1606, of the display system1600. The scan area size corresponds to the size of the FOV area1606, and the scan area location corresponds to a region of one of the lens elements1608,1610at which the FOV area1606is visible to the user. Generally, it is desirable for a display to have a wide FOV to accommodate the outcoupling of light across a wide range of angles. Herein, the range of different user eye positions that will be able to see the display is referred to as the eyebox of the display.