Speckle-reduced direct-retina holographic projector including multiple spatial light modulators

A direct-retina holographic projection system includes first and second spatial light modulators (SLMs) and a control module. The first SLM receives a beam of light and dithers the beam of light at a predetermined frequency to provide multiple instances of the beam of light. The second SLM receives the instances of the beam of light, displays an encoded phase hologram of a graphic image to be projected, and diffracts the instances of the beam of light to provide instances of the encoded phase hologram with the same graphic image but multiplied with dithered wavefronts. The control module: iteratively adjusts a parameter of the first SLM to generate the instances of the beam of light; and controls operation of the second SLM to, based on the instances of the beam of light, display multiple instances of the graphic image on a retina of an eye of a viewer.

INTRODUCTION

The present disclosure relates to direct-retina projection holographic display systems and head up display systems of vehicles.

Display devices are used in a variety of applications. Some example display devices are flat panel displays, projection displays, and head-up displays. Display devices can be either be of a transmission or reflection type.

A driver of a vehicle traditionally views surroundings of a vehicle through windows, windshields, and other glass of the vehicle. The driver may control vehicle acceleration, deceleration, and steering based on the driver's visual observation of the surroundings of the vehicle. The vehicle may include one or more displays that display various information to the driver. For example, some vehicles include an infotainment system that includes a display that displays various infotainment and other vehicle information. The vehicle may also include a head-up display (HUD) that displays information by forming a virtual image at a certain distance with reflection of a windshield. For example, the HUD may display a vehicle speed and other vehicle information (e.g., warnings such as lane departure warnings and collision avoidance warnings).

SUMMARY

A direct-retina holographic projection system is provided and includes a first spatial light modulator, a second spatial light modulator, and a control module. The first spatial light modulator is configured to receive a beam of light from a light source and dither the beam of light at a predetermined frequency to provide multiple instances of the beam of light. The second spatial light modulator is configured to receive the instances of the beam of light from the first spatial light modulator, display an encoded phase hologram of a graphic image to be projected, and diffract the instances of the beam of light to provide instances of the encoded phase hologram with same graphic information as the graphic image but multiplied with dithered wavefronts. The control module is configured to: iteratively adjust a parameter of the first spatial light module to generate the instances of the beam of light; and control operation of the second spatial light modulator to, based on the instances of the beam of light, display multiple instances of the graphic image on a retina of an eye of a viewer by directing the instances of the encoded phase hologram of the graphic image toward a reflector or the eye of the viewer.

In other features, the second spatial light modulator directs the instances of the encoded phase hologram directly at the eye of the viewer without forming a real image on a diffuser and without reflecting the image with mirrors.

In other features, the second spatial light modulator directs the instances of the encoded phase hologram directly at the reflector to be seen by the eye of the viewer without forming a real image on a diffuser. The reflector is a windshield of a vehicle.

In other features, the control module is configured to dither the beam of light, via the first spatial light modulator, at a frequency greater than or equal to 60 Hertz.

In other features, the control module is configured to dither the beam of light between a first angle and a second angle at the predetermined frequency.

In other features, a difference between the first angle and the second angle is less than or equal to 5°.

In other features, a difference between the first angle and the second angle is less than or equal to 1°.

In other features, the first spatial light modulator includes digital light processing microelectromechanical systems mirrors.

In other features, the first spatial light modulator includes a piston mode spatial light modulator.

In other features, the first spatial light modulator includes a ferroelectric liquid crystal spatial light modulator.

In other features, the direct-retina holographic projection system further includes a device driver. The control module is configured to control the device driver to generate one or more driving voltages. The first spatial light modulator changes a dithering angle based on the driving voltages.

In other features, a method of operating direct-retina holographic projection system is provided. The method includes: receiving a beam of light from a light source at a first spatial light modulator; dithering the beam of light at a predetermined frequency to provide multiple instances of the beam of light, where the dithering includes iteratively adjusting a parameter of the first spatial light module to generate the instances of the beam of light; receiving the instances of the beam of light at a second spatial light modulator; converting the instances of the beam of light to multiple instances of an encoded phase hologram; and controlling operation of the second spatial light modulator to, based on the instances of the beam of light, display multiple instances of an image on a retina of an eye of a viewer by directing the instances of the encoded phase hologram toward a reflector or the eye of the viewer.

In other features, the method further includes directing the instances of the encoded phase hologram directly from the second spatial light modulator to the reflector or the eye of the viewer.

In other features, the beam of light is dithered, via the first spatial light modulator, at a frequency greater than or equal to 60 Hertz.

In other features, the beam of light is dithered, via the first spatial light modulator, between a first angle and a second angle at the predetermined frequency.

In other features, a difference between the first angle and the second angle is less than or equal to 5°.

In other features, a difference between the first angle and the second angle is less than or equal to 5°.

In other features, the dithering of the beam of light includes changing driving voltages of digital light processing microelectromechanical systems mirrors of the first spatial light modulator.

In other features, the dithering of the beam of light includes changing a driving voltage of a piston mode spatial light modulator.

In other features, the dithering of the beam of light includes changing a driving voltage of a ferroelectric liquid crystal spatial light modulator.

DETAILED DESCRIPTION

A vehicle may include a holographic HUD that includes a SLM and a coherent or partially coherent light source. The phase hologram is encoded on a SLM. Light from a coherent or partially coherent light source illuminates the SLM that is encoded with a phase hologram and the light is diffracted in a manner prescribed by the hologram. The diffracted light is reflected by a windshield of the vehicle and forms a real image on a driver's retina.

Display systems often employ a source of coherent light, such as a laser, in conjunction with the other display components. Coherent light may refer to light that is spatially and temporally in-phase. A large amount of “speckle” can arise when coherent light is reflected off a diffused surface. When coherent light is reflected from a diffused surface, various points on the surface each emit a light wave. Typically, the reflected light waves have a same frequency, but the phase of the light reflected from different points on the surface can vary resulting in granular, non-uniform intensity due to quasi-random interference. The reflected light interferes constructively and destructively to produce a random pattern of light and dark spots or bands. The overall non-uniform pattern is referred to as “speckle”. When forming an image from the reflected light, the speckles add noise to an image.

Some holographic projection systems reduce speckle by vibrating one or more diffusers or by including and operating an active diffuser, such as a vibrating projection screen. These types of holographic projection systems do not directly project a phase hologram at retinas of a person viewing the generated images without first reflecting the light off a diffuser.

In a direct-retina projection holographic display system, a projected phase hologram is directed to the eyes of a viewer without being reflected off a diffuser. Although removal of a diffuser reduces the amount of speckle in a holographic display system, some speckle can still arise due to reflection of light off, for example, one or more other surfaces. The examples set forth herein include direct-retina projection holographic display systems that do not include a diffuser and are configured to reduce and/or eliminate the amount of speckle. The direct-retina projection holographic display systems include multiple spatial light modulators (SLMs) including a fast-switching SLM for dithering and a second SLM for hologram generation. The second SLM may be liquid crystal on silicon (LCoS). The fast-switching SLM performs as a light dithering SLM that dithers a light beam between predetermined angles and at a high-frequency (e.g., greater than or equal to 60 Hertz (Hz)). This dithering is provided prior to generation of an encoded hologram by the LCoS SLM, which reduces speckle and/or blurring of images as seen by the viewer.

FIG. 1shows an example of a direct-retina holographic projection system110that includes one or more light (or laser) sources (one laser112is shown), a beam expander114, a switching (or first) SLM116, a LCoS (or second) SLM118and a control module120. The laser112generates a laser beam122, which is received at the beam expander114. The beam expander114expands a width of the laser beam122to provide an expanded beam124.

The switching SLM116receives the expanded beam124and iteratively changes an incident angle of the expanded beam124reflected at the LCoS SLM118to provide a dithered expanded beam128. Although primarily referred to herein as a LCoS SLM, the second SLM118may be replaced with a different type of SLM. The dithered expanded beam128includes multiple instances of the expanded beam124. The control module120switches angular states and/or mirror heights of the switching SLM116to change the incident angle of the expanded beam124at a high-frequency (e.g., greater than or equal to 60 Hz). The expanded beam124is dithered to provide multiple beams at respective incident angles referred to as the dithered expanded beam128. The dithered expanded beam128may be viewed as multiple light sources that illuminate the LCoS SLM118at slightly different incident angles.

Example incident vectors {right arrow over (kin,t1)}, {right arrow over (kin,t2)}, {right arrow over (kin,t3)} for three different occurrences of the same beam (or three beams) at three different times are shown to illustrate changes in the incident angle of the expanded beam124. The angle of the vector {right arrow over (kin,t1)} is at an acute predetermined angle (e.g., 45°) relative to, for example, a front surface129of the switching SLM116. The angles of the vectors {right arrow over (kin,t2)}, {right arrow over (kin,t3)} relative to the angle of the vector {right arrow over (kin,t1)} are represented as α1and α2, respectively. The angle α2may be equal and an inverse of angle α1. For example, α2may be −1° and al may be 1°. As another example, the expanded beam124may be dithered ±2.5° from a reference angle (e.g., 45°) or over have a total scanning range of less than or equal to 5°. The angles α1and α2and a difference between the angles α1and α2may be referred to as dithering angles and/or angles defining changes in the projected direction of the expanded beam124from the switching SLM116to the LCoS SLM118. Although three vectors are shown, the expanded beam124may be dithered between any number of angles in any predetermined and/or selected pattern.

The switching SLM116may be implemented as an array of fast-switching spatial light modulators that each have a response time of less than 1 milli-second (ms). As a first example, the array of fast-switching spatial light modulators may include digital light processing (DLP) microelectromechanical systems (MEMS) mirrors. One of the device drivers140controls tilt angles of the DLP MEMS mirrors. The angles are based on the input driving voltages provided by the one of the device drivers140and alter the incident angles of the beam provided to the LCoS SLM118.

As a second example, the switching SLM116may be implemented as one or more piston mode SLMs, where mirror heights of the piston mode SLMs are adjusted by respective input driving voltages. The switching SLM16may include an array of piston mode SLMs and one of the device drivers140of the control module120may provide respective input driving voltages. Changing the heights of the mirrors alters the resultant incident angles of the beam provided to the LCoS SLM118.

As yet another example, the switching SLM116may be implemented as one or more ferroelectric liquid crystal SLMs, which are controlled by one of the device drivers140. The molecular orientation (effective refractive index) of the ferroelectric liquid crystal SLMs are controlled based on an input driving voltages provided by the one of the device drivers140to alter the incident angles of the beam provided to the LCoS SLM118.

The dithered expanded beam128is incident on to the LCoS SLM118. The LCoS SLM118receives a control signal from the control module120, which provides the phase hologram of the graphic to be projected. The diffracted phase hologram beam130from LCoS SLM118is a multiplication of a tilted wavefront and phase hologram of the graphic to be projected in the frequency domain. The output of the switching SLM116is diffracted by the LCoS SLM118and forms an image on the retinas of the viewer. At each instance, due to the slightly different incident angle of the expanded light beam128onto LCoS SLM118, the image formed on the retinas shifts slightly. The shifts in the image occur at the rate of dithering. The rate of dithering may be at a frequency greater than 60 Hz, such that (i) the human eye is unable to detect flicker, and (ii) the slightly shifted images and corresponding speckle are averaged by the retinas of the viewer.

The encoded phase hologram beam130also has multiple different occurrences, which are at different respective reflected angles. Example reflected wave vectors {right arrow over (kr,t1)}, {right arrow over (kr,t2)}, {right arrow over (kr,t3)} are shown for three different occurrences and three different angles of reflection. The three different occurrences correspond directly to the dithering performed by the switching SLM116.

The control module120may include one or more display drivers140. One or more of the display drivers140may adjust the angular states and/or mirror heights of the switching SLM116. Another one or more of the display drivers140may be used to control states of the LCoS SLM118. The display drivers140may be implemented at the control module120and/or at the switching SLM116and/or the LCoS SLM118.

The LCoS SLM118may include a limiting aperture150to mitigate stray light. The limiting aperture150may be implemented as a frame holding the LCoS SLM118.

The encoded phase hologram of the LCoS SLM118may be represented as P(fx, fy), where fx and fy are coordinates in the frequency domain. The encoded phase hologram beam130is outputted from the LCoS SLM118at different angles and received at the eyes (one eye160is shown) of a viewer. The corneal lenses (one corneal lens162is shown) perform an inverse Fourier transform and convert the encoded phase hologram beam130, represented as ei{right arrow over (k)}·{right arrow over (Δr)}P(fx, fy), to a real image p(x,y) seen via the retinas (one retina164is shown) of the viewer.

The dithered expanded beam128, which is in-phase, may be represented as where i is the imaginary number (or square root of −1), {right arrow over (k)} is the incident angle vector and {right arrow over (Δr)} is an optical path length that a wave propagates from the switching SLM116to the LCoS SLM118.

The inverse Fourier transform performed by the eye160may be represented by equation 1.

As an example, equation 1 may be modified to represent the inverse Fourier transform for each of the three different encoded phase hologram beams130out of the LCoS SLM118for the three different times t1, t2, t3. Equations 2-4 provide example representations of these beams.

p2⁡(x,y)=F-1⁢{ei⁢kin,t⁢⁢2→·Δ⁢⁢r→⁢P⁡(fx,fy)}=F-1⁢{ei⁢(kin,t⁢⁢1→·Δ⁢⁢k)→·Δ⁢⁢r→⁢P⁡(fx,fy)}(3)p3⁡(x,y)=F-1⁢{ei⁢kin,t⁢⁢3→·Δ⁢⁢r→⁢P⁡(fx,fy)}=F-1⁢{ei⁢(kin,t⁢⁢1→·Δ⁢⁢k)→·Δ⁢⁢r→⁢P⁡(fx,fy)}(4)
Equations 3-4 provide t1equivalent representations showing ±Δ{right arrow over (k)} (or change in the incident angle vector {right arrow over (k)}). The conversions from t2and t3to t1are represented by equations 5-6, which include wave vectors at different times.
{right arrow over (kin,t2)}={right arrow over (kin,t1)}+{right arrow over (Δk)}  (5)
{right arrow over (kin,t3)}={right arrow over (kin,t1)}−{right arrow over (Δk)}  (6)
Based on the shift theorem of Fourier transform described in equation 7 and equation 8, a tilt in the wave vector k in frequency domain induces the a shift in the projected graphic.
g(x,y)=F−1{G(fx,fy)}  (7)
g(x−a,y−b)=F−1{e−iπ(fxa+fxb)G(fx,fy)}  (8)

The above-described tilting provided by the switching SLM116tilts the wavefront, which induces a shift in the real image p(x,y) formed on the retina. The retina averages the received images, which reduces and/or removes speckle and/or blurring and provides a single image free of speckle and/or blurred areas.

FIG. 2shows an example perspective view from a driver seat of a vehicle200. The vehicle200includes a windshield204located in a front opening of the vehicle200. Passengers within a passenger cabin208of the vehicle200can look through the windshield204to see in front of the vehicle200. While the example of a land-based vehicle is described, the present application is also applicable to air-based vehicles (e.g., airplanes, helicopters, etc.) and water-based vehicles (e.g., boats, etc.). Also, although some examples are disclosed herein with respect to vehicle implementations, the examples are applicable to non-vehicle implementations.

As shown inFIG. 2, the windshield204is visually located above a dashboard206of the vehicle200. The vehicle200may include a steering wheel210. The vehicle200may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

A HUD system300, such as that shown inFIG. 3, projects a hologram212shown inFIG. 2onto a portion of the windshield204through an aperture216in the dashboard204. The hologram212includes various vehicle information, such as a present speed of the vehicle200, a present gear of a transmission of the vehicle200, an engine speed, a directional heading of the vehicle200, present infotainment system settings, and/or other vehicle information. The hologram212presents data to the driver of the vehicle without the driver having to look away from objects in front of the vehicle. As discussed further below, the hologram212includes replicated instances of one hologram as provided by the dithering disclosed herein.

FIG. 3shows a HUD system300that includes a reflector302of a vehicle and the direct-retina holographic projection system100ofFIG. 1with a modified version of the control module120(identified as control module120′). In the example shown, the reflector is implemented as a windshield, but may be a different reflector. The control module120may perform the operations described above and additional operations, such as determining vehicle information to display via the reflector (or windshield302). The direct-retina holographic projection system100includes one or more light sources (one laser112is shown), the beam expander114, the switching SLM116, the LCoS SLM118and the control module120′. The laser112generates a laser beam122, which is received at the beam expander114. The beam expander114expands a width of the laser beam122to provide an expanded beam124.

The switching SLM116receives the expanded beam124and iteratively changes an incident angle of the expanded beam124reflected at the LCoS SLM118to provide a dithered expanded beam128including multiple instances of the expanded beam124. The control module120′ switches angular states and/or mirror heights of the switching SLM116to change the incident angle of the expanded beam124at a high-frequency (e.g., greater than or equal to 60 Hz). The dithered expanded beam128is dithered to provide multiple beams at respective incident angles.

Example incident vectors {right arrow over (kin,t1)}, {right arrow over (kin,t2)}, {right arrow over (kin,t3)} for three different occurrences of the same beam (or three beams) at three different times are shown to illustrate changes in the incident angle of the expanded beam124. The switching SLM116may be configured and controlled as described above.

The LCoS SLM118receives the dithered expanded beam128and based on a control signal from the control module120′ encodes and converts the dithered expanded beam128to provide an encoded phase hologram beam130. The encoded phase hologram beam130also has multiple different occurrences (or instances), which are at different respective reflected angles. Example reflected vectors {right arrow over (kr,t1)}, {right arrow over (kr,t2)}, {right arrow over (kr,t3)} are shown for three different occurrences and three different angles of reflection. The three different occurrences correspond directly to the dithering performed by the switching SLM116.

The control module120′ may include one or more display drivers140. One or more of the display drivers140may adjust the angular states and/or mirror heights of the switching SLM116. Another one or more of the display drivers140may be used to control states of the LCoS SLM118. The display drivers140may be implemented at the control module120′ and/or at the switching SLM116and/or the LCoS SLM118.

The LCoS SLM118may include a limiting aperture150to mitigate stray light. The limiting aperture150may be implemented as a frame holding the LCoS SLM118.

FIG. 4shows a direct-retina holographic projection method which may be implemented by the direct-retina holographic projection system100and the control module120′ ofFIG. 3. Although the following operations are primarily described with respect to the implementation ofFIG. 3, the operations may be easily modified to apply to other implementations of the present disclosure. The operations may be iteratively performed.

The method may begin at400. At402, the control module120′ determines dithering angles and a switching (or dithering) frequency for switching between the angles. This may include determining a switching pattern and/or order of dithering angles, determining mirror tilt angles, and/or determining heights of one or more piston mode spatial light modulators. The dithering angles and/or other corresponding information may be determined based on (i) a predetermined distance between the switching SLM116and the LCoS SLM118, (ii) a predetermined distance between the LCoS SLM118and the reflector302, and/or (iii) a predetermined estimate of a distance between the reflector302and the eyes of a viewer. The other corresponding information may be directly related to the dithering angles and include tilt angles of the beam output from the switching SLM116, mirror tilt angles, heights of piston mode spatial light modulators, etc. The dithering angles are determined to prevent shifted instances of the image projected from being resolved by the viewer. When the instances of the projected image is resolved, the viewer sees a speckled and/or blurred image.

FIG. 5shows a diagram including an example resolved plot500, an example semi-resolved plot502and an example unresolved plot504. A Rayleigh Criterion value a is shown and is directly related to the selected dithering angles of the switching SLM116. The Rayleigh Criterion value a is a distance between peaks of plotted intensity signals. The intensity signals may refer to different instances of the same image. The larger the dithering angle, the larger the Rayleigh Criterion value a and the more resolved the images are to a viewer. The smaller the dithering angle, the smaller the Rayleigh Criterion value a and the more unresolved the images are to the viewer. The control module120′ determines the dithering angles and/or other related information to be in appropriate ranges such that the instances of an image are unresolved. For example, the control module120′ may select the tilt and/or dithering angle of the switching SLM116to be less than a predetermined angle. As the tilt and/or dithering angle increases above the predetermined angle, the instances of an image become semi-resolved.

Referring toFIG. 4, at404, the control module120′ activates the laser112to generate the laser beam122. The laser beam122is directed to the beam expander114, which expands the laser beam122to provide the expanded laser beam124that is provided to the switching SLM116.

At406, the control module120′ controls the dithering of the switching SLM116. The dithering is achieved by fast switching the switching SLM116to direct the expanded laser beam124to slightly different incident angles on SLM118. This may include generating, via one or more of the device drivers140, driving voltages based on the determined dithering angles and the selected switching (or dithering) frequency to alter the tilt angles of the switching SLM116, the tilt angles of the mirrors of the switching SLM116, the heights of the piston mode spatial light modulator(s), etc. The dithering and switching (or scanning) angle is set such that the dithering of a spot formed on retinas of the viewer is unresolvable. The switching SLM116may dither light from a collimated light source, such as the laser112and the beam expander114, and induce multiple illumination incidence angles onto the LCoS SLM118during display of each image frame.

At408, the control module120′ controls operation of the LCoS SLM118to display the phase hologram of graphic to be projected. The LCoS SLM118, with phase hologram of graphic, is illuminated with dithered expanded laser beam124and generates the encoded phase hologram beam130. The encoded phase hologram beam130includes instances of the same phase hologram at different angles as shown inFIG. 3. This may include controlling one or more of the device drivers140to generate driving voltages to control states of the LCoS SLM118.

At410, instances of the encoded phase hologram may be directed at the eyes of the viewer via the reflector302. The reflected holograms with slight different incident angles form slightly shifted images. The speckle of the images are slightly shifted and averaged by the retinas of the viewer, such that the viewer sees a reduced amount of speckle or does not see any speckle.

The phase holograms may be generated based on signals from a vehicle control module. The control module120′ may be implemented as a vehicle control module or may be in communication with a vehicle control module. The control module120′ generates the phase holograms based on vehicle data. The control module120′ may obtain the vehicle data, for example, from a communication bus of the vehicle. The vehicle data may include, for example, the present speed of the vehicle, the present gear of the transmission of the vehicle, the present engine speed, the present directional heading of the vehicle, the present infotainment system settings, and/or the other vehicle information. The method may end at412.

FIG. 6shows an example of the LCoS SLM118ofFIGS. 1 and 3. The LCoS SLM118may be used in any of the embodiments disclosed herein. The LCoS SLM118may include a silicon backplane layer602; a LCoS SLM (or phase modulator) layer including a circuit (or pixelized electrode) layer604, a first alignment layer608, a liquid crystal layer610, a second alignment layer612, and a transparent electrode layer614; and a glass substrate layer616.

The circuit layer604includes control circuitry and/or pixel drivers for controlling the liquid crystal layer610. The circuit layer604may include a transistor for each pixel. Each pixel independently modulates phase of light exiting the LCoS SLM. As an example, if voltages provided to the pixels are different, then phases of light rays out of corresponding portions of the LCoS SLM have different phases. Each of the pixels may have an associated voltage set. The range of the voltages provided to each pixel may vary the phase of the corresponding portion of the phase hologram beam130between, for example, 0-2π to advance or delay the corresponding portion of the light wave coming out of the LCoS SLM118.

The circuit layer604controls the amount and phase of light emitted from the liquid crystal layer610. Orientations of molecules in the liquid crystal layer610and associated with the pixels of the LCoS SLM118change with voltage. The voltage-dependent orientation of molecules induces spatially varying phase distribution on LCoS SLM118. The relation between the amount of phase being modulated and applied voltage can be positive related or negative related, depending on the physical property of liquid crystals. The LCoS SLM layer is further described with respect toFIG. 7. The LCoS SLM118may include a reflective film layer when implemented as a reflective holographic projector.

FIG. 7shows a portion700of a LCoS SLM layer and the control module120′, which may be implemented in the embodiments ofFIGS. 1 and 3. The LCoS SLM layer may include pixels704arranged in an array and connected to drive circuits706,708. The LCoS SLM layer may also include a SLM control module710, which may control the driver circuits706and708. The driver circuits706,708may receive power from the SLM control module710or the control module120′ via switches707,709. The SLM control module710may receive signals directly from wavefront sensors and/or control signals from the control module120′. The control module120′ may receive phase detection signals and control operation of the SLM control module710to adjust voltages provided to the pixels704. In another embodiment, the SLM control module710directly receives the phase detection signals and controls the drive circuits706,708to generate the appropriate voltages, which are applied at the pixels704.

FIG. 8shows the control module120′, one of the device drivers140, and the switching SLM116. As described above, the switching SLM116may include an array of DLP MEMS mirrors800, an array of piston mode spatial light modulators802and/or one or more ferroelectric liquid crystal spatial light modulators804. The device driver140may be implemented as part of the control module120′ or as part of the switching SLM116.