Abstract:
An enhanced resolution microdisplay system using lenses mounted at a distance from the microdisplay, taking advantage of the Talbot effect, which can be produced using arrays of lenses and mirrors. The Talbot effect image of at least one light source created using a fly&#39;s eye lens is imaged onto the microdisplay using a beam splitter, and the combined image is passed by the beam splitter through output optics.

Description:
REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application claims one or more inventions which were disclosed in Provisional Application No. 62/187,325, filed Jul. 1, 2015, entitled “Enhanced Resolution for 2D Images on Microdisplays”. The benefit under 35 USC §119(e) of the U.S. provisional application is hereby claimed, and the aforementioned application is hereby incorporated herein by reference. 
     
    
     BACKGROUND OF THE INVENTION 
       [0002]    Field of the Invention 
         [0003]    The invention pertains to the field of creation of images using a microdisplay device in such a way that the images have more resolution (i.e. more pixels) than the microdisplay itself. 
         [0004]    Description of Related Art 
         [0005]    Microdisplays rely on the formation of spots in different sub regions of the pixels of the microdisplay in sequence as the microdisplay pixels rapidly change in gray scale, so that the image formed is made up of the spots instead of the microdisplay pixels. Projection and head mounted systems that rely on this method for the production of high resolution images are described in DTI patents U.S. Pat. No. 6,734,838 and U.S. Pat. No. 7,417,617, both titled “Enhanced Resolution for Image Generation”. 
         [0006]    U.S. Pat. No. 6,734,838 relies on lenses embedded in the cover glass of the microdisplay in order to place the lenses close enough to the pixels of the microdisplay to allow spots to be focused on the way out of the lenses in order for the enhanced resolution process to work. This configuration would force microdisplay makers to alter their manufacturing process and components to accommodate the cover glass containing embedded lenses. U.S. Pat. No. 6,734,838 also discloses the use of relay lenses to refocus spots formed near a separated fly&#39;s eye lens onto the microdisplay. This configuration is bulky, and requires expensive precision relay lenses. 
         [0007]    The embedded lens approach described above has one major drawback—it requires that the microdisplay manufacturer make custom microdisplays containing the embedded lenses, and spend considerable time and money altering their manufacturing process to accommodate the precise alignment required for the lenses. With some microdisplays, embedded lenses cannot be used at all. Most manufacturers have been unwilling to do this. U.S. Pat. No. 7,417,617 further discloses the use of a birefringent lens which can employ polarized illumination in such a way that it focuses light onto spots on the microdisplay but does not affect the light on the way out, thus allowing the observer to see the spots without magnification or other interference by the lenses when the spots are observed through a correctly oriented polarizer. This doubles the distance that the fly&#39;s eye lens can be placed in front of the microdisplay, and thus allows it to be placed outside the cover glass in some cases. However, some microdisplays can have such small pixels and/or thick cover glass that this technique cannot be used. In cases where it can be used, the lenses are non-standard and difficult to make. 
         [0008]    Periodic lens arrays produce what are known as Talbot Plane images. Talbot Plane images are “copies” of the focal plane images, produced at certain planes beyond the focal plane by the positive interference of the wave fronts emanating from the repeating focal plane images. This effect was described by Besold and Lindlein in an April 1997 article entitled “Fractional Talbert Effect for Periodic Fly&#39;s eye lenses,” Optical Engineering, Volume 36, number 4, pg. 1099-1105. 
         [0009]    These copies are created at greater distances from the lens than the focal plane, and are formed by the constructive interference of light from many different focal plane images. 
         [0010]    Some of these copies will have the same pitch as the original spots as long as collimated light is used to illuminate the lens array. 
         [0011]    When monochromatic illumination sources are used, these copy images have much higher angular resolution than the focal plane images because light from a large number of lenslets, covering a relatively large area, goes into the creation of each of the copies. This greater resolution allows small, sharp spot images to be focused at far greater distances from the lens sheet than are possible using focal plane images—far enough away that the lens could potentially be mounted on the far side of a conventional beam splitting cube. Certain of these Talbot Plane images can be made identical in size, pitch, and spot size to the focal plane image. This allows the lens to be completely out of the way of the light exiting the microdisplay, and allows a conventional fly&#39;s eye lens to be used with an off the shelf, unmodified microdisplay. 
       SUMMARY OF THE INVENTION 
       [0012]    The invention allows the use of conventional fly&#39;s eye lenses mounted at a distance from the microdisplay without the use of relay optics, even if the microdisplay has very small pixels. This configuration would make use of normal, non-birefringent lenses mounted on the far side of the beam splitting optics that are typically used to view reflective microdisplays. The configuration takes advantage of an optical effect known and the Talbot effect, which can be produced using arrays of lenses. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWING 
         [0013]      FIG. 1  shows an illustration of Talbot plane images. 
           [0014]      FIG. 2  shows a block diagram of an enhanced resolution display utilizing Talbot plane images of light spots. 
           [0015]      FIG. 3  shows an example of an image display at full resolution. 
           [0016]      FIG. 4  shows an example of the image of  FIG. 3 , implementing the area of interest tracking method. 
           [0017]      FIG. 5  shows the display of  FIG. 2 , with additional elements to implement the area of interest tracking system. 
           [0018]      FIG. 6  is a flowchart of the area of interest tracking method. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0019]    Talbot plane images are illustrated in  FIG. 1 . 
         [0020]    A microlens array  2 , such as a fly&#39;s eye lens, is illuminated by monochromatic light  1 . For purposes of this discussion, the light  1  is assumed to come from laser light sources in the form of collimated beams filling the fly&#39;s eye lens  2 . The light  1  is focused into images  4  at the common focal plane of the microlenses in the lens array  2 , which in the case of collimated laser illumination will be spots of light. The multiple images formed there generate equal or fractional size copies of themselves in the Talbot planes, which are located at greater distances from the fly&#39;s eye lens as illustrated in  FIG. 1 . This effect is produced by positive and negative interference of light waves exiting the images formed by the fly&#39;s eye lens  2 . 
         [0021]    The Talbot plane images shown in  FIG. 1  are:
       The quarter Talbot plane image  6 , which is a quarter size copy of the primary image  4     The half Talbot plane image  8 , which is a half-size copy of the primary image  4     A series of “n” full sized images, from the first Talbot plane image  10   a  through the “nth” Talbot plane image  10   n.          
 
         [0025]    Of particular interest are the planes known as the first (or “primary”) Talbot plane  10   a  and the half Talbot plane  8 , as shown in  FIG. 1 . Within these planes copies of the focal plane images of nearly the same pitch as the original focal plane images are formed (or of the same pitch in the case of collimated illumination at the normal). 
         [0026]    The distance between the focal plane  4  of the fly&#39;s eye lens  2  and the half  8  or first  10   a  Talbot planes is linearly dependent on wavelength, so in the example above, the planes associated with red, green, and blue wavelengths will be separated from each other by some distance, typically on the order of several millimeters. 
         [0027]    A Talbot plane associated with red light will be closer to the focal plane than the corresponding Talbot plane associated with green light, which in turn will be closer than the corresponding Talbot plane associated with blue light. The distance between the focal plane  4  and the Talbot planes  6 ,  8  and  10   a - 10   n  is also dependent on the pitch of the images in the focal plane  4 , and thus will be different for fly&#39;s eye lens  2  with differing pitches. 
         [0028]    There are three factors which complicate the use of Talbot plane images for this application, and which require variations to the basic design: 
         [0029]    1) The distance between the microlens array  2  and the Talbot planes  6 ,  8  and  10   a - 10   n  is proportionately dependent on wavelength. Red, green, and blue light will produce Talbot plane images at distances that are separated by many millimeters from each other. 
         [0030]    2) Because of factor 1, above, a microlens array designed to work with Talbot plane images must be illuminated with very narrow band laser light. Lasers are the only common illumination sources that are sufficiently monochromatic and bright enough for the application. 
         [0031]    3) Again because of factor 1, above, small changes in laser wavelength with temperature can cause focus distance issues. 
         [0032]    A near-to-eye-resolution microdisplay system using Talbot plane images and which overcomes the disadvantages caused by the three factors listed above is shown in  FIG. 2 . 
         [0033]    Using the system as shown in  FIG. 2 , the resolution of an image created with a microdisplay device  36  is increased by focusing regions of light, usually in the form of circular spots, formed by a fly&#39;s eye lens  28  sequentially into different areas of the pixels of the microdisplay  36 , the microdisplay  36  being spaced substantially apart from the fly&#39;s eye lens  28 . The spots of light originate as an array of images of light sources  20   a - 20   c  that are focused by the fly&#39;s eye lens  28  in a place close to itself, usually within tens of microns of the lenslets of the fly&#39;s eye lens  28 . This array of images are reproduced at or very close to the pixel plane of the microdisplay  36  by means of the Talbot effect, which is described above. 
         [0034]    The fly&#39;s eye lens  28  is a conventional type that when illuminated by monochromatic (laser) light creates an array of very small diffraction-limited spots in a Talbot plane at a large distance (typically 5 mm to 10s of mm) from itself. The distance involved is sufficient to allow such a lens  28  to be mounted on a different side of a beam splitting cube  32  from the microdisplay  36 . 
         [0035]    Red  20   a,  green  20   b,  and blue  20   c  diode laser illumination is ideally used for this type of lens to work properly, since a very narrow bandwidth is required. In addition, the lasers  20   a - 20   c  are preferably of the single mode type and use Bragg reflectors. These types of laser diodes exhibit comparatively little variation in wavelength with temperature. 
         [0036]    The red, green, and blue diode laser illumination light sources  20   a,    20   b,  and  20   c  comes from an array of independently controlled lasers for each color, for example a 2×2 array or a 3×3 array, which collectively constitute the arrays of light sources as described in U.S. Pat. No. 6,734,838 and U.S. Pat. No. 7,417,617. 
         [0037]    Light from the three sets of laser diodes (red  20   a,  green  20   b  and blue  20   c ) is flashed in sequence and directed to the fly&#39;s eye lens  28  by a beam combiner  22  and, optionally, collimating lens  24 . Such beam combiners are readily available off the shelf in the form of dichroic mirror prisms, and are specifically made for the purpose of combining light from red, green and blue light sources into a single beam. 
         [0038]    The light from the light sources  20   a - 20   c  is linearly polarized either through use of a polarizing filter  26  or through use of light sources (such as lasers) that are linearly polarized to begin with. The original light sources  20   a - 20   c  will be large enough that scatter, diffraction, and blockage by the intersection in the beam combiner  22  will not cause any significant effect. 
         [0039]    The fly&#39;s eye lens  28  forms spot images in its own focal plane, very near to the lens  28 , and in Talbot planes at a considerable distance in front of itself. The fly&#39;s eye lens  28  is adjacent to a first side of a beam splitter cube  32  which has four sides  32   a,    32   b,    32   c  and  32   d,  and a partially reflective mirror  37 . The mirror  37  is oriented at 45° to the sides  32   a - 32   d,  so that the mirror  37  passes a portion of light entering the first side  32   a  through to the second side  32   b  opposite the first side  32   a,  and reflects a portion to adjacent third side  32   c.  Similarly, light entering the beam splitter  32  from the fourth side  32   d  is partially passed through to third side  32   c  opposite the fourth side  32   d,  and a portion is reflected to pass out the second side  32   b.  Typically, the partially reflective mirror  37  in a beam splitter  32  will be about 50% reflective and 50% transmissive, but other arrangements would be possible. 
         [0040]    In particular, mirror  37  will ideally be a polarizing mirror, designed to reflect nearly all linearly polarized light exiting the polarizer  26  and fly&#39;s eye lens  28  with a given polarization direction, and to let nearly all linearly polarized light with a perpendicular polarization direction to pass through. Such a mirror will be much more efficient in terms of the use of light than a half reflective mirror, and the following description assumes that a polarizing mirror is present. 
         [0041]    Light from the spots in the focal plane is reflected by mirror  37  of the 45 degree beam splitter  32  toward the collection of stacked dichroic mirrors  30   a - 30   c.  This light first passes through a ¼ wave retarder  66  placed in front of the stacked dichroic mirrors  30   a - 30   c . The first mirror  30   a  is dichroic, and reflects red light but lets the other colors through. The second mirror  30   b  is also dichroic, but reflects green light and lets the remaining blue light through. Blue light passes through the first two mirrors and is reflected by the last mirror  30   c.  The last mirror  30   c  can either be dichroic (reflecting blue light) or simply be a highly reflective broadband first surface mirror, such as an aluminized mirror. 
         [0042]    Light reflected from the stacked mirrors  30   a - 30   c  is reflected back toward microdisplay  36 , most of it passing through mirror  37  to microdisplay  36 . 
         [0043]    Since the three reflective surfaces  30   a - 30   c  are in different positions, the travel distance between the fly&#39;s eye lens  28  and the microdisplay  36  is different for each color. The thicknesses and positions of the mirrors  30   a - 30   c  are chosen so that the travel distance for light of each color is the correct distance to cause its Talbot plane images of all three colors to be focused on the pixels of the microdisplay  36 . 
         [0044]    On passing out of the mirror system  30   a - 30   c  all light goes through ¼ wave retarder  66  again and comes out polarized in the orthogonal direction from which it entered the three-mirror system  30   a - 30   c.  It therefore passes through the beam splitter mirror  32  to the microdisplay  36 . Depending on what type of microdisplay is used, the microdisplay  36  itself will change the polarization of the light again in the course of forming an image, causing the light that forms the image to reflect off the mirror  37  of the beam splitter  32  again, toward output optics  34 , as is known to the art. In the case of a non-polarizing microdisplay, like Texas Instrument&#39;s (TI&#39;s), a second ¼ wave retarder  67  can be added in front of the microdisplay  36  to cause the polarization direction of the light to change to the orthogonal direction when exiting the microdisplay  36 , and thus reflect from the mirror  37  of the beam splitter  32 . 
         [0045]    An optional component can be used, in the form of thermal expansion blocks  68  at the edges of the fly&#39;s eye lens  28 . This is a material that expands and contracts with temperature at such a rate that its expansion changes the spacing between the fly&#39;s eye lens and mirrors and microdisplay in order to compensate for small changes in laser wavelength with temperature. This component would be an alternative to using lasers with Bragg reflectors. 
         [0046]    Although the example above describes a tricolor arrangement, more or fewer colors could be used if an appropriate number of mirrors are provided. For example, four mirrors could be used for a four color system (for example, red, yellow, green, and blue), or two mirrors could be used for a two color system. At the lower limit one fully reflective mirror could be used at a distance in a monochrome system. Note that the mirrors must be in correct order so that shorter wavelength reflecting mirrors are toward the back of the stack. The last mirror can be dichroic or a simple highly reflective broadband first surface mirror. 
         [0047]    The diagram in  FIG. 2  shows output optics  34  as a viewing lens, but these optics could just as easily be a projection lens designed to project the image of the microdisplay  36  and its spots onto a screen spaced apart at a distance from the optics  34 . 
         [0048]    Although a reflective microdisplay  36  is described here, the reflective microdisplay  36  could be replaced by a transmissive microdisplay, provided that the output optics  34  (viewing or projection lens) are moved to a position in front of the microdisplay  36 , on the side opposite the polarizing beamsplitter  32 . 
         [0049]    Although a fly&#39;s eye lens is described, it is also possible, as is explained in U.S. Pat. No. 6,734,838 and U.S. Pat. No. 7,417,617, to use a lenticular lens if resolution increase in only one direction is desired. 
       Incorporating an Area of Interest (AOI) and AOI Tracking Capability 
       [0050]    One interesting aspect of this technology is that it can be easily be used to implement an ultra-high resolution area of interest (AOI) display within a lower resolution peripheral field. 
         [0051]    Such a system is shown in block diagram form in  FIG. 5 . The parts of the display which were previously discussed in respect to  FIG. 2 , above, bear the same reference numbers in  FIG. 5  as in  FIG. 2 , and the previous discussion is incorporated here by reference. 
         [0052]    In  FIG. 5  a user  52  is using the display, and is concentrating on one part of the image being displayed on microdisplay  36  by computer  50 . This point of focus is called the “gaze point”  53 . The computer  50  uses an input from a gaze tracker  51 , which monitors the user&#39;s  52  eye movements as is known to the prior art, to determine the gaze point  53 . 
         [0053]      FIGS. 3 and 4  show an example image demonstrating how the AOI method works.  FIG. 3  shows an example of an image  41  being displayed on the microdisplay&#39;s  36  display area  40 . The user&#39;s gaze point is centered on an area denoted by circle  42 . 
         [0054]    Much computational power and/or data throughput could be saved if the computer  50  only had to render the ultra-high resolution images in the small areas surrounding the user&#39;s gaze point  53  at any time.  FIG. 4  shows the result of this process. 
         [0055]    This much smaller area rendered at near maximum resolution only needs to cover the part of the image  42  around the gaze point  53  where the user can see, plus extra area  46  covering the range of rapid saccadic eye movements and extra area beyond that to ensure overlap between one AOI position and the next during rapid longer range eye movements as the user looks at different objects or details in the scene. The rest of the image  45  can be rendered at lower resolution, as is shown in  FIG. 4   
         [0056]    Fortunately, this is potentially easy using the system described above, and requires no moving parts. Referring to the flowchart of  FIG. 6 , as the microdisplay  36  goes through its refresh cycles, the method works by the computer  50  executing the following steps: 
         [0057]      60 . Calculate a low (or standard) resolution image  41  for the entire display area  40 . 
         [0058]      61 . Display the image  41  on the microdisplay  36  by writing the low (or standard) resolution image  41  onto the pixels for the entire display area  40 . 
         [0059]      62 . Determine the user&#39;s area of interest (AOI)  42  in the image by finding the gaze point  53  using a gaze tracker  51 . 
         [0060]      63 . Determine the additional area  46  around the AOI covering the range of rapid saccadic eye movements and extra area beyond that to ensure overlap between one AOI position and the next during rapid longer range eye movements. 
         [0061]      64 . Calculate a high resolution image for display inside the additional area  46 . 
         [0062]      65 . Write the small, high resolution AOI image with every spot flash within the area of interest  42  and within the additional area  46  around the area of interest  42 . Since the information on the pixels outside the AOI remains the same during the entire (typically 1/60th second) flash cycle, the image will have a resolution equal to the pixel resolution of the microdisplay. 
         [0063]      66 . Repeat the method from step  60 . 
         [0064]    To change the position of the AOI  42 , all that one has to do is change the position of the area  46  where high resolution information is being read onto the microdisplay  36 . One can do this purely in software, without any mechanical movement. 
         [0065]    Assuming that the computer  50  can calculate and display sixty high resolution AOI inserted images per second, the lag caused by this part of the system will be a maximum of one microdisplay refresh cycle, typically about 1/60th second, between any two successive AOI positions, no matter how widely separated they are. In addition, it may be beneficial to move the AOI from position to position during less than 1/60th second refresh cycle, for example during each 1/540th field, displaying different details of a moving image as it goes along and the eye follows it. It also may be possible, with some microdisplays, to run the microdisplays at a significantly faster rate than 60 flash cycles per second, potentially leading to even less lag. 
         [0066]    Accordingly, it is to be understood that the embodiments of the invention herein described are merely illustrative of the application of the principles of the invention. Reference herein to details of the illustrated embodiments is not intended to limit the scope of the claims, which themselves recite those features regarded as essential to the invention.