Image enhancement for three-dimensional displays

A three-dimensional (3D) display apparatus includes an image generation subsystem (IGS) for generating at least one two-dimensional (2D) image and providing filtered rays derived from the at least one 2D image to an optical element for modulating the filtered rays and producing a 3D image, wherein the filtered rays comprise limited overlap of pixel information. A method and a computer program product for generating 3D images are provided.

BACKGROUND OF INVENTION

1. Field of the Invention

The teachings herein relate to three-dimensional (3D) displays and, in particular, to image enhancement for a 3D display.

2. Description of the Related Art

Optical elements that use spatial de-multiplexing have been used to create 3D images for nearly a century. Exemplary optical elements include narrow vertical slits or lenticular sheets.

The method of spatial de-multiplexing by use of lenticular sheets (i.e., arrays) generates a 3D image from a two-dimensional (2D) image. The resulting 3D image has a lower resolution than that of the 2D image. Reference may be had toFIG. 1.

In a prior art display apparatus10depicted inFIG. 1, an image source2provides a 2D array of pixels3. The 2D array of pixels3provides information to a single array of lenticular lenses4(in some embodiments, the array of lenses is referred to as a “lenticular lens”). The lenticular lenses4optically manipulate light from pixels3to generate ray information5that forms a 3D image wavefront6.

The 3D wavefront6is produced by aligning each lenticule of the array of lenticular lenses4so that a field of view for the lenticule covers several pixels in the array of pixels3. Each pixel that is within the field of view is converted to ray information5by the refractive properties of the lens. Thus, each lenticule becomes an emitter with angularly varying intensity components. The geometrical properties of the 3D wavefront6can consequently be represented by correct parameterization of spatially and angularly varying components of the ray information5. In general, the more pixels3that can be associated with the lenticular lenses4, the more accurate the 3D image wavefront becomes. Thus, in the prior art, it is desirable to have a very high resolution image source2, since this will allow one to produce a 3D image with modest spatial resolution and good ray sampling. However, such display apparatus10are not without drawbacks.

For example, such display apparatus10typically use cylindrical lens arrays as the lenticular lenses4. The use of cylindrical lens arrays creates horizontal-parallax-only (HPO) imagery. One particular drawback of using cylindrical lens arrays is that in the common configuration, horizontal resolution and vertical resolution of the image source2are not equally reduced by the lenticular lenses, resulting in a 3D image6with unequal horizontal and vertical resolution. Reference may be had toFIG. 2.

InFIG. 2A, a portion of the prior art display apparatus is depicted. This illustration shows cylindrical lenticular lenses4which are aligned with the image source2. The alignment of the lenticular lenses4is consistent with an orientation of the pixels3in the image source2. That is, as shown in this illustration, the lenticular lenses4share a direction (a y-axis direction) with the pixels3included in the array. The orientation of the lenticular lenses4are further described inFIGS. 2B and 2C. InFIG. 2B, the lenticular lenses4are shown according to the x-axis (from the top), and inFIG. 2C, the lenticular lenses4are shown according to the y-axis (from the side). As shown inFIGS. 2B and 2C, the image source2produces output rays5which are focused by the lenticular lenses4. Whereas the reconstructed 3D image6has one reconstructed vertical pixel for each input pixel3, the reconstructed x resolution is degraded in this illustration by a factor of approximately ⅕.

InFIG. 2, the lenticular lenses4are shown as generally cylindrical lenses. As the cylindrical lenses do not have any optical power along their height (the y-axis), they cannot convert pixel information into ray information along this axis (seeFIG. 2C). In a typical 3D display, this type of arrangement sacrifices quality of the 3D image along one axis.

InFIG. 3A, an image source2provides an array of pixels3. As shown in the illustration, the pixels3are distributed along an x-axis and a y-axis. As is known in the prior art, lenticular lenses are used to provide for focusing of output rays from the image source2. In this example, the lenticular lenses are tilted lenticular lenses30.

In this example, the tilted lenticular lenses30are tipped slightly with respect to the pixels3of the image source2. This provides an effect such that a respective center of each pixel3for each row of pixels3is offset slightly from an optical axis31of each tilted lenticular lens30. Since a direction for an output ray5from each pixel3is proportional to offset from the optical axis31, adjacent pixels3along a column are output in different ray directions. Thus, vertical resolution may be controlled along with the horizontal resolution to achieve greater ray sampling. That is, in practice, an observer will see a multiple of the number of rays (for example, 8 instead of 4) as compared to the system using parallel lenticular lenses. This is shown inFIG. 3D. Although this technique provides for equalization of resolution in the x-axis and the y-axis, output rays5from pixels3of adjacent rows overlap one another, and thus produce the equivalent of a low-pass filter. Reference may be had toFIGS. 3B,3C and3D as well asFIG. 4.

InFIG. 3B, a first row of pixels in the image source2is shown. InFIG. 3C, a second row of pixels in the image source2is shown. The second row is slightly offset from the first row (that is, effectively offset from the first row), due to the tilting of the lenticular lens30. Output rays5from the combination are shown inFIG. 3D. The x-axis depicted inFIG. 3shows an alignment of pixels3in each row with the pixels in the first row (Row1).

Various forms of focusing lenses are used. In this example, the tilted lenticular lenses30having a desired orientation are generally referred to as “tilted” or “clocked.” The parallel lenticular lenses4(which have no angular deflection from a column of pixels) are generally referred to as “parallel” (with reference to a pixel axis) and by other similar terms. It is recognized that lenticular lenses may take on various forms and are not limited to tilted, parallel, cylindrical, elliptical or by other shapes. Each lenticule may be a singlet, doublet, or other lens type.

FIG. 4shows how providing the tilted lenticular lenses30(relative to a prior art cylindrical lenticular lens4) provides a desired view resolution. Though centers of pixels3for pixels3in adjacent rows are horizontally offset, the horizontal field of view for these pixels overlap, effectively low-pass filtering the output rays5and producing a generally undesirable effect. This results in “view overlap” (interview crosstalk), which has a side effect of limiting the degree of depth in the reconstructed 3D image6. The resulting view overlap between two pixels in adjacent rows is shown inFIG. 4C.

What are needed are methods and apparatus, such as those disclosed herein, for providing improved resolution in a 3D image, where a vertical resolution and a horizontal resolution (along both axes of a 2D image source) appear to be about equal, and where view overlap is minimized.

BRIEF DESCRIPTION OF THE INVENTION

Disclosed is a three-dimensional (3D) display apparatus, the apparatus including: an image generation subsystem (IGS) for generating at least one two-dimensional (2D) image and providing filtered rays derived from the at least one 2D image to an optical element for modulating the filtered rays and producing a 3D image, wherein the filtered rays comprise limited overlap of pixel information.

Also disclosed is a method for producing a three dimensional (3D) image, the method including: providing a display apparatus including an image generation subsystem (IGS) for generating at least one two-dimensional (2D) image and providing filtered rays derived from the at least one 2D image to an optical element for modulating the filtered rays and producing the 3D image, wherein the filtered rays comprise limited overlap of pixel information; and modulating the filtered rays for producing the 3D image.

Further disclosed is a computer program product stored on machine readable media, the product including instructions for producing a three dimensional (3D) image by executing instructions including: operating a 3D display apparatus including an image generation subsystem (IGS) for generating at least one two-dimensional (2D) image and providing filtered rays derived from the at least one 2D image to an optical element for modulating the filtered rays and producing the 3D image, wherein the filtered rays comprise limited overlap of pixel information; and producing the 3D image.

DETAILED DESCRIPTION OF THE INVENTION

Disclosed is a three-dimensional (3D) display apparatus that provides for images produced from at least one two-dimensional (2D) image. The 3D display provides generally uniform resolution images. Since it is typically desirable to have equal horizontal and vertical resolutions, a technique for equalizing resolution equally along both axes of the 2D image is provided. Since it is typically desirable to minimize view overlap in reconstructed 3D imagery, a technique for minimizing view overlap is provided.

Referring now toFIG. 5, there are shown aspects of a 3D display apparatus100according to the teachings herein. The 3D display100includes an image generation subsystem68(IGS). Typically, the IGS68includes an image source2(as an example, an LCD panel) and a mask60. In general, and as in the prior art, the image source2produces an image from a plurality of pixels3. The pixels3are arranged, for convention, along an x-axis and a y-axis (as an example, refer toFIG. 3A). Included as a part of the IGS68is a mask60.

The mask60receives output rays5from the image source2and filters the output rays5. Thus, it may be considered (at least for convention herein) that the mask60produces filtered rays61. The mask60is typically adapted to receiving output rays5from the image source2and providing filtered rays61useful for the optical element included in the display apparatus100. That is, the mask60is typically adapted for the particular optical element (e.g., the parallel lenticular lens4or the tilted lenticular lens30, or any other type of optical element included in the 3D display100).

In an embodiment, the mask60is a two-dimensional barrier such as that illustrated inFIG. 7A, in which a region opaque to visible light is depicted in black, and regions transmissive to visible light are depicted in white. Each light-transmissive area (referred to as a “portion of pixel62”) has a size and shape that is a function of several factors. Exemplary factors include, without limitation, a number of ray directions reconstructed by the 3D display, a distance between adjacent lenticules (the “lenticular pitch”), and a resolution density along the x-axis and the y-axis. For example, a 3D display in a standard 3:4 ratio may have an image diagonal measuring 20″ (50.8 cm), corresponding to a horizontal width of 16″ (40.6 cm) and a height of 12″ (30.5 cm). A 25-view 3D image with a 5×5 pattern of portion of pixels62and a per-view resolution of 1024 horizontal pixels×768 vertical pixels requires an image source2with 1,024×5=5,120 pixels horizontally by 768×5=3,840 pixels vertically. Therefore, in the horizontal direction, each pixel3has a pitch of 5,120 pixels/406 mm=12.6 pixels/mm. This equals a pixel width of 79 microns, and by similar calculation, a pixel height of 79 microns. In order to reconstruct25ray directions from a square region of 25 pixels, in this example, the filtered rays61are generated by diagonally staggered portions of pixels as illustrated inFIG. 7A; the width of each portion of pixel is ⅕ the width of a pixel3, and the height of each portion of pixel is equal to the height of a pixel3. That is, each portion of pixel62is approximately 15.8 microns wide and 79 microns high.

The mask60can be made in a variety of processes, including standard printing processes onto transparent film that is attached to a transparent carrier, glass, a diffuse surface, or a lenticular array using electrostatic or chemical adhesive means.

In other embodiments, the mask60includes a one dimensional barrier. The filtered rays61are directed to the lens array. In this example, the lens array includes an array of vertical lenticular lenses30in order to produce the 3D wavefront6.

The term “filtered rays” generally refers to the ability of the 3D display100to provide output rays5in substantially separated bundles. That is, by one standard, the filtered rays61do not overlap, or appear to have overlap (to the unaided eye of a human observer). Thus, the production of filtered rays61as disclosed herein results in the 3D image that appears to have improved resolution over the prior art, and does not include blurring effects, or those effects that result from overlap of pixel information.

The quality of 2D displays100, such as spatially-multiplexed multi-view 3D displays100(e.g. lenticular, parallax, barrier, etc.), is improved by, among other things, limiting the crosstalk between views. In various embodiments, limiting crosstalk between pixels3is accomplished by including the mask60or an aperture design into the 3D display100. In some embodiments, special pixel architectures are included directly in the image source2, obviating the need for the mask60. Optics used in the display100may include a wide range of optical systems, including those in which the lenticular lenses (or other optical demultiplexing elements) are “clocked” relative to the orientation of the pixel columns in the source image2.

The 2D image source2is considered to be at least a part of an “image generation subsystem” (IGS)68. The IGS68may include a variety of embodiments. For example, in some embodiments, the mask60is included, in others the mask60is excluded. Exemplary and non-limiting embodiments foregoing the mask60include embodiments where the image source2provides each pixel3according to a pattern that is essentially provides the respective portion of pixel62(reference may be had to the special architectures discussed above). That is, the teachings herein provide for use of image sources2that are arranged for producing output rays5that are the equivalent of the filtered rays61, and providing the equivalent rays directly to an optical element for the 3D display100. Typically, commercially available image sources2are used.

An exemplary image source2includes for example, a liquid crystal display (LCD). Other types of image sources2include, without limitation, a cathode ray tube (CRT), a vacuum fluorescent display (VFD), a digital light processing display (DLP), a plasma display panel (PDP), a light-emitting diode (LED), an organic light-emitting diode (OLED), a surface-conduction electron-emitter display (SED), a field emission display (FED), and a liquid crystal on silicon display (LCOS), or any source which produces an image which may be projected at or near the lenticular array, such as onto an intermediate diffuse surface. Of course, one skilled in the art will recognize that certain adaptations may be called for to apply such technologies to the teachings herein.

In various embodiments, aperture arrays, masks60and other such components may be included in the IGS68. More details regarding an optical mask60are now provided.

Some embodiments of the mask60include a vertical barrier. In these exemplary and non-limiting embodiments, the vertical barrier is composed of opaque and transparent lines. For a25view display with 400 μm lenticular lenses, the magnified DMD pixel will be 80 μm. To reduce pixel crosstalk the vertical barrier will aperture each 80 μm pixel to 25 μm. Thus the vertical barrier has a line pair width of 80 μm where the opaque line is 55 μm thick and the transparent line is 25 μm thick. The vertical barrier can be made on glass with chrome or emulsion. The glass can range in thickness from 100 μm to 3000 μm. The emulsion or chrome layer can range in thickness from 0.050 μm to 1 μm. In one example, the vertical banier is provided by Advance Reproduction (Andover, Mass.) and is made on film with a total thickness of 177.902 μm. The emulsion thickness is 1 μm. The vertical barrier can either be vertical to the film's edge and then the lenticular screen is tilted at a 11.3 degree angle (from vertical) which corresponds to a ⅕th slope allowing for 25 separate views in the horizontal direction. The alternative is to etch or print the vertical lines at the 11.3 degree angle while keeping the lenticular screen square to the viewing plane (i.e., vertical). To align a tilted vertically striped barrier (i.e., mask60) with precision on the lenticular screen usually calls for precision alignment marks (100 μm or larger) along the border of the etched film or glass. These marks can be aligned carefully with markings on the lenticular screen.

Various embodiments of optics may be used, for example, to equalize resolution along the x-axis and the y-axis of the image source2. As used herein, tilted lenticular lenses30are an embodiment of such optics, as a parallel lenticular lenses4. Other types of optics may be used to provide the desired effect. Accordingly, use of the term “tilted lenticular lenses” and “parallel lenticular lenses” are merely exemplary and are not limiting of the teachings herein.

A result of these various embodiments is that pixel data does not overlap in an output 3D image. Reference may be had toFIG. 6. InFIG. 6AandFIG. 6B, the IGS68provides portions of respective pixels3. The output rays5that result do not include any ray overlap, as shown inFIG. 6C.

When the pixel structure referred to inFIG. 6is used in conjunction with a parallel lens array4(i.e., one that is vertically aligned to a pixel grid of the image source2), better ray sampling results than by use of the tilted lenticular lens30alone. Thus, more realistic 3D imagery can be displayed.

Referring now toFIG. 7AandFIG. 7B, two embodiments of the IGS68are shown. InFIG. 7, desired pixel structures for enhanced 3D image quality and equal vertical and horizontal resolution reduction are provided. In each of the patterns depicted inFIG. 7, a grid of five (5) pixels by five (5) pixels is masked to produce a pattern of twenty five (25) white rectangles as shown. For each pixel3, the white rectangles depicted correlates to a portion of the pixel62. By employing a technique that includes use of the mask60, the grid of pixels may be transformed to a single pixel3in the 3D wavefront6. Thus, horizontal resolution and vertical resolution are equalized and ray overlap is avoided.FIG. 7Bdepicts a desired pixel structure, which corresponds to a degree of tilt in the tilted lenticular lens30.FIG. 7Ashows a near approximation ofFIG. 7B, which is produced by overlaying a 1D barrier pattern over a regular pixel grid.

InFIG. 8, the pixel structure of the IGS68allows for greater amount of rays to be modulated than embodiments of 3D displays using either one of parallel lenticular lenses4and tilted lenticular lenses30with regular grid pixel structures.

As used herein, “modulating” refers to one or more of: refracting, transmitting, aperturing, reflecting, diffusing, diffracting, or performing any other optical manipulation.

FIG. 9provides another embodiment of a pixel structure that makes use an IGS68having a regular grid of rectangular pixels and a tilted lenticular lens array30. In this case, a horizontal component of the tilted lenticular lens array30includes the desired ray information.

FIG. 10illustrates another embodiment of the 3D display100. InFIG. 10, the 3D display100includes an enhanced-resolution 2D display20and 3D image projection optics220. 3D imagery is projected into a volume approximately indicated by a 3D image volume300and is visible by an observer180across a horizontal field of view (wherein, the field of view is a function of the 3D image projection optics220).

The 3D image projection optics220include a diffuser250that scatters the illumination from an enhanced image surface21(i.e., the pixels3) to an inter-view barrier260. The inter-view barrier260(i.e., the mask60) passes modulated light to a lenticular lens array280that is mounted between an upstream glass plate270and a downstream glass plate290. The lenticular array can be any of a number of well-known lenticular arrays, or can be replaced with an optical element that serves the purpose of optically demultiplexing pixels3at the enhanced image surface21to a variety of ray trajectories toward one or more observers. Alternatives to the lenticular lens array280include a parallax barrier array, a multi-element holographic optical element, and a fly's-eye lens array (for full-parallax 3D images), or other arrays as discussed herein.

One of the simplest methods for producing source imagery with the desired pixel structure is to manufacture an image source2(in the case of dynamic imagery) or printing process which inherently uses the desired structure to present the imagery. However, most conventional printing processes and image sources2are manufactured with a regular pixel grid. Therefore, an efficient technique for producing the desired pixel structure includes using the light blocking mask60with a desired pattern over the image source2, where the image source2uses a regular pixel grid. For manufacturing simplicity, a simple barrier pattern can be used to nearly approximate the desired pixel structure. When a barrier is used as the mask pattern, it is also possible to change the configuration such that the barrier is aligned to the pixel grid, and the lenticular lenses are tilted with respect to the barrier and grid.