Patent Description:
Virtual head-up displays (HUD) are employed in aircraft, land vehicles, retail store windows to present a person/user with information overlaid onto immediate surroundings. Many vehicle HUDs make use of the inside surface of the windshield as an optical combiner to provide the user a 2D or 3D stereoscopic image of any relevant information to be delivered.

An important issue with traditional HUDs is that they lack abilities such as software-based aberration correction and eyebox adjustments. Aberration correction, in itself allows for a larger field-of-view (FOV) to be cast across a larger eyebox, albeit no single optical component can be designed to form a large FOV aberration-free image due to the fact that information radiating from the display is aberrated as it is reflected from the windshield of the vehicle. A holographic HUD setup has many advantages over conventional HUD applications, such as the case where with a holographic head-worn device, a wide field of view (FOV) is attainable and aberration correction, as well as interpupillary distance are dynamically adjustable.

Dynamic holography includes the spatial light modulator (SLM) as an active optical element, which are devices that are dynamically programmable to implement <NUM>-dimensional complex multiplicative optical transparencies that are functions of two spatial coordinates and time. In holographic display applications, SLMs are generally deployed to display computer-generated holograms (CGHs). Existing SLM technology in the art is commonly based on liquid crystal technology, liquid crystal on silicon (LCoS) technology, MEMS-based digital micromirror array technology to name a few. LCD SLMs are transmissive, whereas LCoS SLMs are reflective in principle. Transmissive SLMs based on liquid crystals have larger pixel pitch arising from the electric circuitry associated therewith having been embedded within the pixel aperture as a corollary. On the other hand, reflective LCoS SLMs can be made to have much smaller pixel pitches since it is possible to bury electronics under the pixel(s) in question. Although SLMs are ideally expected to provide full complex modulation, practical SLMs provide only some restricted type of modulation, such as phase-only, amplitude-only, binary modulation etc. A multitude of algorithms are designed such as the Gerchberg-Saxton, iterative Fourier and error diffusion in order to encode a desired full-complex hologram into a restricted form hologram. These procedures and applications generally result in the emergence of noise along with signal beams. Another practical issue with SLMs is that most SLMs do not possess <NUM>% modulation efficiency, that is, only a fraction of the incident light gets modulated by the SLM, while rest of the light remains unmodulated. Almost all SLMs are pixelated devices, resulting in the generation of higher diffraction order replicas of signal, noise and unmodulated beams. In the case of holographic HUD designs, only the main signal beam should enter the eye and reach the retina, while beams from noise and unmodulated beams as well as higher diffraction order replicas should be blocked. This requirement necessitates additional measures.

One of the prior art publications in the technical field of the present invention may be referred to as <CIT>, which teaches a sharp foveal vision combined with low resolution peripheral display with a wide field-of-view (FOV) and a rotatable hologram module capable of creating a high-resolution steerable image. In another document, US <CIT>, a near-to-eye display device including an SLM, a rotatable reflective optical element and a pupil-tracking device are disclosed. The pupil-tracking device tracks the eye pupil position of the user and based on the data provided by said pupil-tracking device, the reflective optical element is rotated such that the light modulated by the spatial light modulator is directed towards the user's eye pupil.

<CIT> discloses a head-up display device comprising a light-emitting image source along with optical elements that form a beam path. Optical elements comprise a holographic optical element with an optical imaging function and a reflector. Said reflector and the holographic optical element are arranged so that beams emitted by the former into a third section of the beam path can at least partly transilluminate the holographic optical element, wherein illumination angles of transilluminating beams in the third section of the beam path substantially deviate from angles of incidence at which part of the imaging function of the holographic optical element becomes effective.

<CIT> and <CIT> discloses a windscreen having spatially variant optical power likely to result in distortions, wherein the display has a shaped diffuser to compensate for the distortions of the windscreen and a holographic projector for projection of images thereon. The holographic projector has an SLM arranged to display a hologram representative of the image and apply a phase delay distribution to incident light, wherein the phase delay distribution is arranged to bring the image to a non-planar focus on the diffuser. The HUD may have a mirror with an optical power, or parabolic curvature, to redirect light from the diffuser onto the windscreen. In another aspect of application, a method of compensating for the spatially variant optical power of a windscreen is provided using the apparatus above wherein a virtual image is formed using the windscreen. Further relevant prior art are documents <CIT> and <CIT>.

Primary object of the present invention is to provide a holographic HUD device consisting of at least one light module that is capable of providing virtual images at multiple depths.

Another object of the present invention is to provide a holographic HUD device with steerable exit pupils across an exit pupil plane.

A further object of the present invention is to provide a holographic HUD device having at least one SLM, where corrections of aberration and interpupillary distance are calculated on at least one computing means and implemented on the SLMs to increase image quality and achieve large FOV. A still further object of the present invention is to provide a holographic HUD device that utilizes beam steering simultaneously to deliver rays to both eyes of a user.

A still further object of the present invention is to provide a holographic HUD device which optical steering is utilized on two exit pupils separated by an adjustable interpupillary distance.

A still further object of the present invention is to provide a holographic HUD device which utilizes partially real-time rendering for stereographic holography.

It is to be noted that in the present invention, holographic stereograms are the means for producing a three-dimensional effect. In one embodiment of the present invention, different, separate SLM modules are responsible for steering of beams to each eye box of a user.

According to one embodiment of the present invention, a holographic head-up display generated on the surface of vehicle windshield proves advantageous as it is effective in correcting aberration so as to limit spread and therefore enhance control thereof, as well as what is provided with transmissive and reflective elements and combinations thereof in different embodiments.

According to an embodiment of the present invention, an eye-tracking system for pupil tracking projection assembly based on a camera system relays information concerning pupil position to a processing means with the aim of interpupillary distance calculation, following which said IPD outcome is used for adjustment of computer-generated hologram through simultaneous beam steering for both eyes of a user.

According to one embodiment of the present invention, spatial positioning of the steering mirror after the imaging lens forms a hierarchy between components, which possess optical power, and components, which do not.

Accompanying drawings are given solely for the purpose of exemplifying an object reconstruction system, whose advantages over prior art were outlined above and will be explained in brief hereinafter.

The drawings are not meant to delimit the scope of protection as identified in the claims nor should they be referred to alone in an effort to interpret the scope identified in said claims without recourse to the technical disclosure in the description of the present invention. The drawings are only exemplary in the sense that they do not necessarily reflect the actual dimensions and relative proportions of the respective components of any system or subsystem.

The following numerals are referred to in the detailed description of the present invention:.

According to the present invention, a device and a system in the form of a computer-generated holographic projection display device and a system comprising thereof is proposed. More specifically, a device and a system for holographic head-up display (HUD) projection seen through a vehicle windscreen is proposed.

Referring to <FIG>, the holographic HUD (<NUM>) comprises of holographic projection module (<NUM>) containing the optical system and electronics, optical steering apparatus (<NUM>) aimed to create a steerable eyebox in front of the driver's eyes, a head tracker camera (<NUM>) for tracking the driver's head motion, face, and pupils, and a head tracking control (<NUM>) system. Other input from external sensors and vehicle's sensors as well as the input from the head-tracking control (<NUM>) are analyzed at the vehicle computer (<NUM>) and the proper content is calculated to be shown at the HUD (<NUM>) system. The driver sees the virtual image (<NUM>) at the distance determined by the holographic HUD (<NUM>). CGH can contain multiple virtual images (<NUM>) that appear at different depths to the user's eye (<NUM>).

Referring to <FIG>, the HUD (<NUM>) device optics form exit pupil(s) (<NUM>) at the exit pupil plane (<NUM>) and a virtual SLM image (<NUM>) formed by the imaging lens (<NUM>) appears behind the windscreen (<NUM>). Light module (<NUM>) consists of at least one from each of the following components: SLM (<NUM>), light source (<NUM>), source lens (<NUM>) for beam shaping and fold mirror. The figure shows a cross-sectional view. One light module (<NUM>) is needed for each eye of the user. In this embodiment steering mirror (<NUM>) is after the imaging lens (<NUM>), which results in smaller footprint for the beam on the steering mirror (<NUM>). The field-of-view (FOV) of the system can be measured from the exit pupil plane (<NUM>) to the footprint on the windscreen (<NUM>). For a fixed FOV, rotation of the steering mirror (<NUM>) moves the exit pupil (<NUM>) location without increasing the size of the optical beam on the imaging lens (<NUM>).

<FIG> shows another embodiment, wherein the steering mirror (<NUM>) and the intermediate image plane (<NUM>) of the SLM (<NUM>) coincides. The imaging lens (<NUM>) appears after the steering mirror (<NUM>). Virtual SLM (<NUM>) plane is the optical conjugate of the intermediate image plane (<NUM>). When the steering mirror (<NUM>) plane coincides with the intermediate image plane (<NUM>), rotation of the steering mirror (<NUM>) does not change the location of the virtual SLM plane (<NUM>). Spatial filter (<NUM>) plane is an optical conjugate of the exit pupil plane (<NUM>). Instead of the steering mirror (<NUM>), SLM (<NUM>) or the entire light module (<NUM>) can be rotated for moving the exit pupil (<NUM>) location across the exit pupil plane (<NUM>).

Referring to <FIG>, a head-up display device (<NUM>) is provided comprising at least two light modules (<NUM>), each of which further comprise at least one light source (<NUM>) together with an SLM (<NUM>); in the form of a computer-generated hologram display device seen through a vehicle windshield (<NUM>) according to at least one embodiment. Device and system disclosed in the present invention rely on display of said computer-generated holograms to spatially modulate light incident from said at least one light source (<NUM>). Computer-generated holograms are displayed on an interior surface of a vehicle windshield (<NUM>) as a means for peripheral interaction and retrieval of information regarding the surroundings and the vehicle in question (i.e. navigation/map, gauges such as speedometer, and other dashboard elements). System disclosed in the present invention, comprising the HUD device also disclosed in the present invention; proposes a configuration characterized with achieving a large field-of-view (FOV) and compliance with the concave surface(s) of vehicle windshield(s) (<NUM>) in question. Windshield (<NUM>) can be a wedge shaped windshield with varying thickness across in order to steer ghost reflection from the back surface away from the exit pupil(s) (<NUM>).

<FIG> shows a general schematic view of a holographic HUD device (<NUM>) comprising a light source (<NUM>), two light modules (<NUM>), imaging lens(es) (<NUM>) and a spatial filter (<NUM>). Light source (<NUM>) is followed by the illumination lenses (<NUM>), which can be located before or after the SLM (<NUM>) and deliver rays to the spatial filter (<NUM>) plane.

Referring to <FIG>, HUD basic optical system architecture uses a spatial filter (<NUM>) to block the undesired beams (not shown in the figure) generated by the SLM (<NUM>) and let the desired modulated beams (<NUM>) (the beams that would provide the visual information to the viewer within the exit pupils (<NUM>)) reach the exit pupil plane (<NUM>). Two light modules (<NUM>) - one per eye - are utilized to form an initial copy of the exit pupils (<NUM>). The visual information is generated by the computer-generated holograms displayed on the SLMs (<NUM>). Each light module (<NUM>) images the at least one point light source (<NUM>) onto the spatial filter (<NUM>) plane, where for each module, the light distribution is essentially given by the Fourier transform of the SLM transmittance function, apart from trivial multiplicative phase factors. In another embodiment the HUD may have a single light module for both eyes with two point light sources (one for each eye). The undesired beams (<NUM>) - the unmodulated beam, noise beam, and higher order replicas - get spatially separated in the spatial filter (<NUM>) plane, and hence can be filtered out with apertures that let only the desired beam to pass unaffected. In <FIG>, the optics module is implemented as a simple <NUM>-f telescope. In an actual design, it should be noted that this module can be any imaging system that images the source to the spatial filter plane (151c), and may include reflective, refractive, multi-part, conventional, diffractive, freeform components, some of which may be used off-axis and/or to introduce folds. Likewise, SLM (<NUM>) is illustrated as a transmissive component, but it can be reflective component. In a different embodiment, off-axis illumination directly from the light source (<NUM>) or a waveguide plate can be used to illuminate the SLM (<NUM>). Waveguide plate can be used to couple light in and out of the waveguide, which guide the light using total internal reflection.

The spatial filter (<NUM>) plane, consisting of the apertures that only pass the signal beams for left and right eye, gets imaged to the actual exit pupil plane (<NUM>) where the eyes of the viewer would be present. That imaging is performed in the figure by the imaging lens (<NUM>). The imaging may in general perform a non-unity magnification. Most likely, it will be desired that the optics modules residing in the back side of the system occupy minimum possible volume, so that the copies of the exit pupils (<NUM>) on the spatial filter plane (151c) are much closer to each other than the typical human interpupillary distances (<NUM>). In such cases, magnification of the imaging system would be greater than unity and the imaging system can cause optical distortions and aberrations. In this figure, the imaging between spatial filter (<NUM>) and exit pupil planes (<NUM>) is accomplished with a single imaging lens (<NUM>). In an actual design, it should be noted that this lens can be replaced with an arbitrary imaging system that may include reflective, refractive, conventional, multi-part, diffractive, freeform components, some of which may be used off-axis and/or to introduce folds. In the figure, the virtual image (<NUM>) observed by the viewer is first formed as a real or virtual image (<NUM>) on the intermediate image plane (<NUM>). This image is mapped to the final virtual image (<NUM>) by the imaging lens (<NUM>). Note that the location of the intermediate image plane (<NUM>) depends on the distance of the virtual object plane from the user. For a 3D virtual content, the intermediate image planes (<NUM>) for each virtual object plane form a continuum. In this architecture, the CGHs on the SLMs are not Fresnel holograms. The CGH for each virtual object point occupies only a sub-region of the SLM aperture. In some designs, the SLM is conjugate to a virtual image (<NUM>) plane. In that case, the CGH essentially resembles the image itself, apart from distortions and possible multiplicative phase terms for aberration correction on the exit pupil plane (<NUM>).

As illustrated in <FIG>, at least one pointing laser beam, preferably an infrared laser beam, can be part of the light module (<NUM>) and provide a substantially focused tracking spot (<NUM>) at the exit pupil plane (<NUM>). The tracking spot (<NUM>) or multiple tracking spots (<NUM>) can easily be detected by the head tracking system (<NUM>) and provide automatic calibration for finding the user's eye (<NUM>) to direct the exit pupil (<NUM>) towards the user's eyes (<NUM>).

Referring to <FIG>, in an alternative embodiment, two SLMs (<NUM>) are directly imaged on the exit pupil plane (<NUM>). If no spatial filtering is applied to the light emerging from the SLM (<NUM>), all the undesired beams (<NUM>) - unmodulated beam, noise beams, higher order replicas - will contribute to the SLM images (<NUM>) formed on the exit pupil plane (<NUM>), and lead to undesired ghost images, contrast reduction, bright spots in the background. An angle selective optical filtering means (<NUM>) placed after the SLM can be used to eliminate the undesired beams (<NUM>), so that the SLM images (<NUM>) on the exit pupil plane (<NUM>) are formed only by the information bearing signal beams.

The angle selective optical filtering means (<NUM>) may be implemented with holographic optical elements (HOE) that have intrinsic angle and wavelength selectivity, or with conventional optics that uses prisms, apertures, or a combination. In this embodiment, the CGHs on the SLMs are Fourier holograms. The CGH for each virtual object point occupies the entire SLM aperture. Apart from multiplicative phase terms, the CGH is essentially given by the Fourier transform of the desired image.

Referring to <FIG>, in some embodiments, the HUD optics form virtual images (<NUM>) of the SLM (<NUM>) that lie beyond the imaging lens (<NUM>). In the figure, the virtual SLM images (<NUM>) are aligned so that the centers of left and right virtual SLM images (<NUM>) coincide. In other embodiments, there may be an offset in between, as well to increase the perceived field of view. In the illustrated embodiment, real images of the SLMs are also formed at a plane between the light modules (<NUM>) and imaging lens (<NUM>). That plane also corresponds to an intermediate image plane (<NUM>) as well. In some embodiments, a steering mirror (<NUM>) may be placed at that location, so the location of the exit pupils (<NUM>, left) and (<NUM>, right) on the exit pupil plane (<NUM>) can be moved together when the steering mirror (<NUM>) rotates.

Referring to <FIG>, the HUD (<NUM>) optics form real image of the SLMs (<NUM>) on the viewer side of the imaging lens (<NUM>). In the figure, the SLM images (<NUM>) are aligned so that the centers of left and right SLM images (<NUM>) coincide. In other embodiments, there may be an offset in between, as well to increase the perceived field of view. In the illustrated embodiment, real images of the SLMs are also formed at a plane between the imaging lens (<NUM>) and exit pupil plane (<NUM>). In some embodiments, a steering mirror (<NUM>) may be placed at that location, so the location of the exit pupils (<NUM>) on the exit pupil plane (<NUM>) may be moved together. The real SLM images (<NUM>) may also form on or after the exit pupil plane (<NUM>) in some alternative embodiments.

Referring to <FIG>, holographic projection module (<NUM>) provides illumination to the optical steering means (<NUM>), which is illustrated with a scanning mirror or a steering mirror (<NUM>) in the current embodiment. As the user's eye (<NUM>) moves to different locations shown as <NUM>-A, <NUM>-B, <NUM>-C, head tracking system (<NUM>) detects the new position of the user's eye (<NUM>) and the steering mirror (<NUM>) is deflected to positions <NUM>-A, <NUM>-B, and <NUM>-C.

Referring to <FIG> (top view) and <FIG> (side view), a steering mirror (<NUM>) effectively rotates the virtual space lying behind it around its axis of rotation. Rotation of the steering mirror (<NUM>) can cause rotation of the virtual objects as well. Therefore, in general, the CGHs need to be calculated for each new exit pupil (<NUM>) position. Correct perspective images need to be rendered according to the location of the users' left and right eyes (<NUM>, left) and (<NUM>, right) and their positions. In particular cases, where the steering mirror (<NUM>) is conjugate to an object plane (such as illustrated by <FIG>), the virtual object placed on the virtual image (<NUM>) plane remains stationary, regardless of the rotation of the steering mirror (<NUM>). In <FIG>, the spatial filter (<NUM>) plane is an optical conjugate of the light source (<NUM>) and the exit pupil plane (<NUM>). Given the distances illustrated in <FIG> and assuming the imaging lens (<NUM>) has an effective focal length of f, the following relationships are satisfied in the current embodiment. <MAT> <MAT>.

Referring to <FIG>, the steering mirror (<NUM>) is placed at a plane between the imaging lens (<NUM>) and exit pupil plane (<NUM>). In such cases, the required mirror clear aperture size will be large, but required tilt angles will be small. Also the imaging lens (<NUM>) will be small.

In <FIG>, the steering mirror (<NUM>) is placed at a plane between the spatial filter (<NUM>) plane and the imaging lens (<NUM>). In such cases, the required clear aperture of the steering mirror (<NUM>) will be smaller, but required tilt angles will be larger. Note that for the same field of view, the required clear aperture size for the imaging lens (<NUM>) for <FIG> is smaller in comparison to that of <FIG>. Smaller clear aperture provides important additional advantages as it reduces the aberrations caused by the imaging lens (<NUM>) or lenses and reduces the overall volume of the HUD (<NUM>) optics.

<FIG> illustrates a typical dashboard image to be displayed on HUD (<NUM>) and <FIG> illustrates the associated procedure for CGH computation. Part of the dashboard data consists of speedometer, engine RPM, temperature, time readings, and logos. This part of the data is generated from a limited finite set of possibilities. To speed up the CGH computation, the base CGHs (the part of the CGH excluding final phase term multiplications) for such parts can be precomputed and stored to some memory in the hologram computation processor or the vehicle computer (<NUM>), and the appropriate CGH can be retrieved based on instantaneous cruise data. Other parts of dashboard data, such as navigation maps, video feeds from side view mirrors, rear mirror, and backing cameras or mirror image of a smart phone main screen, need to be generated from a much larger set of possibilities and they are referred to as dynamic parts. The corresponding CGH for the dynamic parts need to be computed real-time. For this part, real time computation can utilize computational resources such as GPUs, FPGAs, ASICs. The computation, involving possible iterations such as free-space propagation (FSP) and iterative Fourier transform algorithms (IFTA), can be carried with simulation windows covering merely the size of the related sub-images, i.e., the simulation window pixel count does not need to be as large as the SLM pixel count. Once base CGHs for precomputed parts are retrieved and for dynamical parts are calculated, dashboard image size base CGH can be obtained. These can be appropriately combined, and multiplied with possible phase terms to get the final CGH as illustrated in <FIG>.

A holographic HUD (<NUM>) can compensate for hardware modifications in a system merely with changes in computational procedure, without additional hardware changes. <FIG>, illustrates case <NUM> for a HUD system designed for a flat windshield (<NUM>). The CGH is computed such that a diffraction limited exit pupil (<NUM>) forms on the exit pupil plane. (<NUM>) Case <NUM> is illustrated in <FIG>, where the windshield (<NUM>) is curved. Keeping the same CGH as in case <NUM> results in an aberrated exit pupil (<NUM>). <FIG> illustrates case <NUM>, the aberrations due to curved windshield (<NUM>) in case <NUM> are compensated by appropriately modifying the CGH of case <NUM>. As illustrated in <FIG>, multiplication with an additional phase term, which may be a Zernike function, would be sufficient to compensate for the aberrations and to obtain an aberration-free exit pupil (<NUM>). In some cases, more advanced procedures may be required, such as applying pre-distortion on the target image, and modifications of the point spread function on the SLM plane.

Referring to <FIG> and <FIG>, the holographic HUD (<NUM>) system can provide software based IPD adjustment by modifying the CGH computation. <FIG> shows a system with a predefined nominal IPD (<NUM>). As illustrated in <FIG>, when the CGHs generating the nominal IPD (<NUM>) are altered appropriately, the IPD value may be changed. The simplest way of steering the exit pupil (<NUM>) location on the exit pupil plane (<NUM>) is to multiply a CGH with a linear phase term that would cause an additional angular deflection on the SLM (<NUM>) plane. In some cases, such as when the virtual image (<NUM>) plane is conjugate to the SLM (<NUM>) plane, this simple solution may be sufficient. In other cases, merely using linear phase terms may result in aberrations forming on the exit pupil plane (<NUM>), and distortions appearing on the observed virtual image (<NUM>). In such cases, additional Zernike polynomial terms may be included on top of the linear phase terms.

<FIG> illustrates various optical filtering means (<NUM>) to eliminate undesired beams (<NUM>) while letting desired modulated beam (<NUM>) that carries the image information. Undesired beams (<NUM>) can be the unmodulated <NUM>th order beam, higher diffraction orders due to the pixelated nature of the SLM, and undesired conjugate beam and undesired replicas of the desired modulated beam (<NUM>) created by the pixelated and imperfect nature of the SLM (<NUM>). <FIG> shows the spatial filter (<NUM>) based method of elimination. <FIG> illustrates the use of holographic optical elements (HOE), which transmit the desired modulated beam (<NUM>) while reflecting the undesired beams (<NUM>). <FIG> illustrates separation of the desired modulated beam (<NUM>) using a prismatic element and total internal reflection (TIR) at one interface to separate beams with different propagation directions. Undesired beams (<NUM>) can be <NUM>% reflected due to TIR.

<FIG> illustrates the desired modulated beam (<NUM>) and the undesired beams (<NUM>) for different illumination colors. The light distribution at the spatial filter (<NUM>) or the Fourier filter plane for a holographic display system. When the SLM (<NUM>) is illuminated with red, green, and blue color light sources (<NUM>). The corresponding patterns appear at the following boxes in the figure: Red is in boxes with circle corners, green is in boxes with diamond corners, and blue is in boxes with star shaped corners. The desired modulated beam (<NUM>) window or the signal window can not have an arbitrary size. Its size is at most equal in both directions to the distance between <NUM>th and <NUM>st orders of the unmodulated blue beam. Some embodiments may allow for both vertical and horizontal noise bands. In some embodiments, no vertical noise band is placed (signal window occupies the whole width from <NUM>th to <NUM>st of unmodulated blue beam) and only horizontal noise bands can be allowed, and vice versa.

In some embodiments, the Fourier filter is an aperture which selects one of the quadrants around the 0th unmodulated beam as illustrated in <FIG>. The desired modulated beam (<NUM>) or the signal beam is placed within the indicated rectangular aperture. If the eye pupil also matches the size of the corresponding exit pupil (<NUM>) on the exit pupil plane (<NUM>), this system can merely support a fixed exit pupil (<NUM>) location, that can not be further steerable with modification on CGHs such as multiplying with linear phase terms. If the eye pupil size is much smaller than the size of the corresponding exit pupil (<NUM>) on the exit pupil plane (<NUM>), the actual desired modulated beam (<NUM>) carrying the signal information would be steerable within the signal window with CGH modifications. However, such configurations will result in poor utilization of the available space bandwidth product (SBP) of the SLM (<NUM>), where SBP a measure for the information capacity an optical system possesses and proportional to the number of pixels in the SLM (<NUM>). Efficient utilization of SBP requires the instantaneous exit pupil (<NUM>) size not to be much greater than viewers eye pupil size.

Referring to <FIG>, in some embodiments, the Fourier filter is an aperture in the form of a horizontal band that lies below (above) the <NUM>th order unmodulated beam. Although part of the region can be uniquely addressed and designed as the desired modulated beam (<NUM>) window, this window can be steered within the horizontal band as in <FIG>. If the user's eye (<NUM>) pupil acts as a second spatial filter (<NUM>) on the exit pupil plane (<NUM>) that selects only the desired modulated beam (<NUM>) window and filter out the higher order replicas and noise, the system is able to deliver a ghost-free image and noise free image. In some embodiments, all the computational noise can be placed outside the passband of the spatial filter (<NUM>), and hence all the noise will get blocked. Coupled with the head tracking system (<NUM>) and pupil tracker, such spatial filtering provides an opportunity for software based IPD adjustment and exit pupil (<NUM>) steering. This software based exit pupil (<NUM>) steering can be used in combination with the optical steering apparatus (<NUM>) such as acousto-optic scanners (<NUM>) and steering mirrors (<NUM>).

In some embodiments, the spatial filter (<NUM>) consists merely of opaque dots blocking the unmodulated beams. Coupled again with a pupil tracker, and again assuming that the viewer's pupil acts as a secondary spatial filter (<NUM>) to select the desired modulated beam (<NUM>) window (or signal window) and eliminate higher order replicas and noise, this configuration gives the ability for exit pupil (<NUM>) steering over a much larger area across the exit pupil plane (<NUM>), theoretically, without any limit. But in practice, the brightness decays rapidly as the signal window gets away from the <NUM>th order unmodulated beam. Limiting the location of the signal window within the first orders of the unmodulated beam (the <NUM> quadrants around the <NUM>th unmodulated beam) is practically a reasonable choice. As long as the signal beams do not concentrate significant energy on the opaque dots, the presence of the opaque dots will be unrecognizable by the viewer. Referring to <FIG>, which shows a case where the desired modulated beam (<NUM>) window or the signal window is steered to a location containing the opaque dots. As long as the eye pupil of the user fits within the signal window, and energy is not concentrated on the opaque dot, the opaque dot will be equivalent to a minor obscuration on the pupil of the user, which will be unrecognizable if it is much smaller than the user's eye (<NUM>) pupil.

<FIG> shows a <NUM>-axis rotatable steering mirror (<NUM>) structure using two electromagnetic actuated motors attached at the backside of the mirror. The configuration is designed to minimize the inertia of the steering mirror (<NUM>) structure. In an alternative embodiment, one can use a double gimbal structure. The actuator motor and its controller should be designed to provide vibration immunity.

<FIG> shows SLM (<NUM>) tiling options using <NUM>, <NUM>, <NUM>, and <NUM> SLMs (<NUM>). As illustrated in the figure, SLMs (<NUM>) can be tiled horizontally and vertically and their orientation can be adjusted based on the aspect ratio of SLMs (<NUM>). SLM tiling can be seamless or some inter-SLM spacing might be introduced between the active areas of two adjacent SLM (<NUM>). <FIG> shows tiling of two SLMs, side by side, without any seam in between using a beam splitter (BS) (<NUM>). SLM position can be adjusted to introduce overlap, no overlap, or seamless tiling of the SLM (<NUM>) and the virtual SLM (<NUM>). Distance of each SLM (<NUM>) to the BS (<NUM>) can be adjusted to remove any phase errors between tiled SLM (<NUM>) and the virtual SLM (<NUM>). BS (<NUM>) can be a polarizing BS and quarter wave plates can be used in front of the SLMs to change the polarization of the light for light efficient combination.

Referring to <FIG>, each eye optical module consists of collimation and focusing lenses. Light source (<NUM>) with different color, i.e., red, green, and blue (RGB) light sources, can be spatially separated or combined with a beam combiner. RGB color filters (151c) at different spatial locations can be introduced at the spatial filter (<NUM>) plane to separate out the desired modulated beam (<NUM>) and undesired modulated beam (<NUM>) for each color. Polarization optical components such as polarizer (<NUM>) and half wave plate (<NUM>) can be used to select the desired polarization and rotate the polarization in the desired direction before the beam is sent to the subsequent optics and the windscreen (<NUM>).

Referring to <FIG>, SLM (<NUM>) has an active area with active pixels. Subsections of the SLM (<NUM>) referred to as 13R, <NUM>, 13B can be dedicated to different sub holograms. Such subsections can have color filters or color filtering can be performed at the spatial filter (<NUM>) using the red, green, and blue color filters at spatial filter plane (151c). Subholograms can be large sections or can be at pixel level as illustrated in <FIG>. In another embodiment, color filters can be replaced with a pixelated liquid crystal shutter device that is electrically controlled to open and close different pixels to let or block light in different sections of the spatial filter (<NUM>) plane.

Referring to <FIG>, an acousto-optic scanner (AOS) (<NUM>) can be used to steer the optical beam and act as an optical steering apparatus (<NUM>). AOS can steer the beam in different directions depending on the frequency of the signal applied to the acousto-optic medium. By changing the frequency, first order angle can be changed. As shown in <FIG>, it is possible to steer the beam in a range of angles by applying a chirped acoustic wave to the acousto-optic medium.

Referring to <FIG>, a foveated display (<NUM>) combines central display (<NUM>) with small FOV and peripheral display (<NUM>) with large FOV. Peripheral display (<NUM>) might be formed using a projector that illuminates a transparent holographic screen attached to the windscreen (<NUM>). Since the peripheral display (<NUM>) image appear on the windscreen (<NUM>), user's eye (<NUM>) need to focus on the windscreen (<NUM>) in order to see a sharp image for the peripheral display (<NUM>) content. When the user's eye (<NUM>) is focused on the virtual image (<NUM>) provided by the central display (<NUM>) or the holographic projection module (<NUM>) in this invention, the peripheral display (<NUM>) image appears blurred as illustrated in the figure.

Referring to <FIG>, when user's head is straight and within the eyebox, exit pupil (<NUM>) for left eye and right eye illustrated with the crosses and user's eyes (<NUM>) are well aligned. When the user's head tilts while using the display, exit pupil (<NUM>) may no longer be aligned with both of the user's eye (<NUM>). As illustrated in <FIG>, exit pupil (<NUM>) for one eye can be moved vertically by changing the location of the window that contains the desired modulated beam (<NUM>). Such software-based adjustment by adding a grating phase term on the CGH effectively compensate for the head tilts. Amount of vertical movement required in the spatial filter (<NUM>) plane is the amount of vertical shift required at the exit pupil plane (<NUM>) divided by the optical magnification of the system. As discussed above, horizontal adjustments of the exit pupil (<NUM>) allow for IPD (<NUM>) adjustments.

Referring to <FIG>, head tilt can be compensated by moving one eye light module (<NUM>) relative to the other eye light module (<NUM>). <FIG> demonstrate moving the exit pupil (<NUM>) vertically for one eye using two fold mirrors, where one of the mirrors is movable as illustrated. Vertical up motion of the fold mirror results in the vertical down movement of the corresponding exit pupil (<NUM>).

Referring to <FIG>, a magnifying prism pair (<NUM>) can be used to change the magnification of the imaging system in one axis. Such a magnification change results in effectively changing the aspect ratio of the display. As an example, two <NUM>:<NUM> aspect ratio SLMs (<NUM>) can be tiled horizontally to achieve <NUM>:<NUM> aspect ratio. Using a magnifying prism pair (<NUM>) to increase the vertical magnification by <NUM> times results in an aspect ratio of <NUM>:<NUM> or approximately <NUM>:<NUM>.

In one aspect of the present invention, a head-up display device (<NUM>) comprising at least one light module (<NUM>) wherein each light module (<NUM>) consists of at least one light source (<NUM>) and at least one spatial light modulator (<NUM>), which display computer generated holograms to spatially modulate the light incident from said at least one light source (<NUM>) is proposed.

In another aspect of the present invention, said at least one light module (<NUM>) form desired modulated beam (<NUM>) that carries holographic image information and undesired beams (<NUM>); wherein undesired beams (<NUM>) are blocked with optical filtering means (<NUM>); while the desired modulated beam (<NUM>) transmitted through the optical filtering means (<NUM>) form at least one exit pupil (<NUM>) on an exit pupil plane (<NUM>) for viewing the head-up display content; and wherein the exit pupils (<NUM>) created by each of the at least one light module (<NUM>) are steerable across the exit pupil plane (<NUM>) using an optical steering apparatus (<NUM>).

Claim 1:
A head-up display device (<NUM>) comprising at least one light module (<NUM>) wherein each light module (<NUM>) consists of at least one light source (<NUM>) and at least one spatial light modulator (<NUM>), which display computer generated holograms to spatially modulate the light incident from said at least one light source (<NUM>) characterized in that;
said at least one light module (<NUM>) form desired modulated beam (<NUM>) that carries holographic image information and undesired beams (<NUM>); wherein undesired beams (<NUM>) are blocked with optical filtering means (<NUM>); while the desired modulated beam (<NUM>) transmitted through the optical filtering means (<NUM>) form at least one exit pupil (<NUM>) on an exit pupil plane (<NUM>) for viewing the head-up display content,the exit pupils (<NUM>) created by each of the at least one light module (<NUM>) are steerable across the exit pupil plane (<NUM>) using an optical steering apparatus (<NUM>);and wherein said optical steering apparatus (<NUM>) comprises rotatable steering mirror (<NUM>), said steering mirror (<NUM>) executes steering for both left eye exit pupil (<NUM>) and right eye exit pupil (<NUM>) across the exit pupil plane (<NUM>) together.