Patent ID: 12210318

DETAILED DESCRIPTION OF THE INVENTION

This application claims priority to U.S. Provisional Application No. 62/137,325 (Smalley; TILEABLE, COPLANAR, FLAT-PANEL HOLOGRAPHIC DISPLAY AND HAPTIC INTERFACE OR HOLODECK PANEL), filed Mar. 24, 2015.

This disclosure incorporates several documents by reference: U.S. Patent Publication 20090040294 (Smalley et al., HOLOGRAPHIC VIDEO DISPLAY SYSTEM); U.S. Patent Publication 20120050833 (Bove, Jr. et al., METHODS AND APPARATUS FOR HOLOGRAPHIC ANIMATION); U.S. Patent Publication 20140300694 (Smalley et al., ANISOTROPIC LEAKY-MODE MODULATOR FOR HOLOGRAPHIC VIDEO DISPLAYS); U.S. Patent Publication 20140300695 (Smalley et al., FULL-PARALLAX ACOUSTO-OPTIC/ELECTRO-OPTIC HOLOGRAPHIC VIDEO DISPLAY); and U.S. Pat. No. 8,149,265 (Smalley et al., HOLOGRAPHIC VIDEO DISPLAY SYSTEM).

In one embodiment, the invention disclosed herein is a tileable and scalable unit capable of generating a 3-D display, directed audio, and a tactile field.

FIG.5illustrates a preferred embodiment of the disclosed invention, comprising at least 3-D display510and slit plane530.

Multiple technologies and/or approaches may be used to construct or generate 3-D display510. For example, several patent publications and issued patents disclose details for generating a 3-D display: U.S. Patent Publication 20090040294; U.S. Patent Publication 20120050833; U.S. Patent Publication 20140300694; U.S. Patent Publication 20140300695; and U.S. Pat. No. 8,149,265.

In a preferred embodiment, 3-D display510may be generated by a holographic display with light deflected by one or many surface acoustic wave patterns. Other 3-D display concepts may include, but are not limited to, lenticular, barrier line, LCD, MEMS, LCOS, ferroelectric, coded aperture, micropolarizer, view sequential, waveguide directional, panoramagram, frustrated total internal reflection, liquid lens, backlight steering/eye-tracked, coupled electro-optic, flying fiber, nanophotonic array, nanoantenna tuned laser, and holographic images.

3-D display510may be constructed as a display stack, i.e., as a set of stacks which each perform various functions for 3-D display510. The stack layers of 3-D display510could include, for example, a monolithic flat-panel holographic video display, a spacer layer, and a lenticular array.

FIG.4illustrates an exemplary slit plane410that may be used in a preferred embodiment of this invention. Slit plane410may be made of multiple materials, including but not limited to copper-clad fiberglass or ceramic, aluminum including anodized aluminum, plastic, metal, wood, stone, glass, single crystal, polycrystalline or amorphous silicon or silicon nitride, single crystal or ceramic or other types of piezoelectric materials or piezo-metal sandwiches. Slit plane410may also be made from a combination of these or other materials. These materials may serve as substrates for both passive and active, electronic, optical, acoustic, and MEMs (microelectromechanical) devices.

In a preferred embodiment, slit plane410may be capable of supporting or integrating ultrasonic transducers420a . . . n. The material may also support electronic busses, integrated optics, and MEMs devices antennas for driving the ultrasonic transducers. In a preferred embodiment slit plane surface450may absorb light. This may allow slit plane410to spatially filter noisy light from within, i.e., behind, slit plane410as well as reduce glare from room light or sunlight outside, i.e., in front of slit plane410. The material may be used to provide acoustic impedance matching and an optimal material may also have an acoustic absorbing material on the back side. Exemplary light-absorbing materials may include, but are not limited to, black paint, lacquer, plastic, metals such as aluminum and steel anodized with dark dyes. Slit plane410may also absorb sound or acoustic energy. Acoustic absorbers may include, but are not limited to, resins, polymers, meshes, foams, and other known acoustic absorbing materials. Slit plane410may be of uniform thickness, or may be of varying thickness, or may be of varying sizes. In a preferred embodiment, slit plane410may have width and height dimensions to match the dimensions of 3-D display510.

In one embodiment, slit plane410may be transparent or semi-transparent. Slits430a . . . nmay be made of transparent material which may have transparent conductors on the surface made of transparent conductive material such as Indium Tin Oxide (ITO). Ultrasonic transducers420a . . . nmay be made of optically transparent plastic membrane and transparent electrodes. By utilizing transparent materials the active, light illuminating regions can be maximized. A fully transparent slit plane would not impede light at all and would obviate the need for lenslet array layer520. The front layer (formally, the slit layer) would just serve as the substrate layer for ultrasonic transducers and traces. If transparent materials are used for the display backplane, the whole display could be made to be transparent.

Slits430a . . . nare disposed in slit plane410so that the light550a . . . nfrom 3-D display510passes through slits430a . . . n. In a preferred embodiment, slit plane410is designed to be placed directly on or in front of 3-D display510, as illustrated inFIG.5. In general, the number, location, orientation, size, and spacing of slits430a . . . ndepend on the particular characteristics of 3-D display510. In one exemplary embodiment, 3-D display510may emit light550a . . . nthat is focused vertically through slits430a . . . n, leaving the 3-D horizontal information unaffected. In some embodiments, vertical 3-D information may also be included as long as it can pass through slit430n. In these embodiments, the slit shape and dimensions may be modified as necessary to accommodate vertical 3-D information. The light emitted from the 3-D display, which passes through slits430a . . . n, enters in through the sides of the slit in a manner that minimizes the need for beveling the sides of the slit. In this embodiment, the number and density of slits430a . . . nmay be as low as two per display and as many as one per hogel (“holographic element”). The positioning of slits430a . . . non slit plane510is arbitrary. In one embodiment, slits430a . . . nmay be uniformly distributed across the area of slit plane410. Although positioning of slits430a . . . nis arbitrary, some factors may guide or affect slit positions in various applications. For example, ability to control sound increases as the density of slits430a . . . nincreases. Also, if the slits are placed a half wavelength apart (wavelength of the ultrasound) then the acoustic field can be controlled at any viewing angle.

Slit plane410may also serve as a spatial filter by blocking scattered light directed at areas on slit plane410other than slits430a . . . n, thereby increasing signal-to-noise ratio and reducing glare. Scattered light may be blocked on the inner side of slit plane410, i.e., the side facing 3-D display510. Room light may be absorbed, thereby reducing glare, by the outer side of slit plane410, i.e., the side away from 3-D display510.

Slit dimensions, i.e., height and width, are generally arbitrary depending on the needs or characteristics of a particular application. If the length of slit430n(i.e., the longer dimension) is shorter than half the wavelength of the illumination light of 3-D display510, significant scattering could result. Such scattering could, in some applications, actually be a feature, as it may increase the vertical view angle for HPO displays. Slits430a . . . ncould be shaped other than as a rectangle, including but not limited to circles or annuli. A person of ordinary sill will be familiar with the characteristics, benefits, detriments, and various applications, that are associated with various slit shapes.

Slit dimensions may be subject to or limited by needs for transducer wiring and mounting requirements. In general, as the dimensions of a slit430nincrease, the ability of slit plane510to act as a filter may decrease. In a preferred embodiment, the width of slit430nmay be roughly equal to the circle-of-least-confusion or width of the point-spread function of lenslet array520, i.e., slit430nmay be as wide as the smallest width of the focused point.

In one embodiment, slit plane410is oriented as shown inFIGS.4and5, and slits430a . . . nmay be oriented horizontally, i.e., with the long edge parallel to the ground or the bottom of slit plane410. In general, horizontal orientation of slits430a . . . nmay provide imagery for viewing along the horizontal axis. Horizontal orientation of slits430a . . . ndoes not prevent vertical parallax. Slits430a . . . nmay be all oriented horizontally, or may be all oriented vertically, or may be a combination of horizontally and vertically oriented slits. Different orientations of slits430a . . . nmay provide imagery for viewing along different axes, which may be a desirable or useful feature for some applications. Slits430a . . . ncould also be oriented other than purely horizontally or vertically, e.g., vertically or at other angles, depending on particular circumstances or applications.

In some embodiments, slit plane410could be oriented such that the face of slit plane410, i.e., the plane with slits430a . . . n, is horizontal relative to the floor, e.g., oriented like the surface of a table top. The aspects of this invention apply to such an orientation, or to any other orientation.

A person of ordinary skill will recognize that the number, location, orientation, size, spacing, and any other characteristics of slits430a . . . nmay change, but that slits430a . . . nare sized, positioned, and oriented in slit plane410so that the light550a . . . nfrom 3-D display510passes through slit plane410unhindered, or substantially unhindered, and 3-D display410continues to substantially function. For example, in some applications using different 3-D displays, slits430a . . . nmay be oriented vertically, horizontally, diagonally, or in some combination of vertical, horizontal, and diagonal. The number, location, orientation, size, and spacing of slits430a . . . ndepends, in general, on the characteristics of 3-D display510.

The shape of one or more of slits430a . . . nmay vary depending on the properties or characteristics of 3-D display510. For example, in one embodiment, to accommodate a particular 3-D display510, one or more of slits430a . . . nmay be in the shape of a cross. Other shapes are possible depending on the properties or characteristics of 3-D display510.

In a preferred embodiment, slit plane410is monolithic.

Also, in one embodiment, the walls of one or more slits430a . . . nmay be angled. The “wall” of slit430nis the sides of the slit as slit430nextends through slit plane410. For example, to accommodate light from 3-D display510, the wall of slit430na slit may be angled so that a dimension of a slit increases or decreases moving from the back, i.e., the side facing 3-D display510to the front, i.e., the side away from 3-D display510, of slit430n, or vice versa, or in any other manner to accommodate light from 3-D display510.

Ultrasonic transducers420a . . . nare well-known in the art. In general, an ultrasonic transducer is a device that converts ultrasound waves to electrical signals or vice versa. As is known in the art, ultrasonic transducers may be used, among other things, to generate a tactile field or to generate directed audio. For example, see U.S. patent application Ser. No. 14/149,518 (“Method and apparatus for providing tactile sensations”), U.S. Patent Publication 2015/0192995 (“Method and apparatus for providing tactile sensations”), and WO2016007920A1 (PCT/US2015/040045, “Three dimensional tactile feedback system”) for details on using ultrasonic transducers to generate a tactile field. See also, Watanabe, Toshio, and Shigehisa Fukui, “A method for controlling tactile sensation of surface roughness using ultrasonic vibration.”Robotics and Automation,1995, Proceedings.,1995 IEEE International Conference on, Vol. 1. IEEE, 1995; Hoshi, Takayuki, et al. “Noncontact tactile display based on radiation pressure of airborne ultrasound.”Haptics, IEEE Transactions on, Vol. 3, No. 3 (2010): 155-165.

Applying ultrasonic transducers to generate directed audio is also well-known in the art. See, for example, U.S. Pat. No. 8,369,546 to Pompei (“Ultrasonic Transducer for Parametric Array”); EP0973152 (Appl. 19990305632 19990715) to Pompei, (“Parametric audio system”); and U.S. Patent Publication No. 20160014529 (“Transparent Parametric Emitter”). These documents are incorporated herein by reference.

FIG.5shows an exploded view500of 3-D display510and slit plane410, as combined in a preferred embodiment of this invention.FIG.5also shows lenslet array520, which may be placed between 3-D display510and slit plane410. In one embodiment, lenslet layer520focuses the light from 3-D display510vertically through slit plane410. Ultrasonic transducers420a . . . nmay be attached, adhered, connected, or otherwise secured to, or included or manufactured as a part of, front of slit plane410. As shown inFIGS.4and5, the front (i.e., outside) of slit plane410is the side of the slit plane opposite, i.e., away from, 3-D display510. The “back” or “inside” of slit plane410is the side opposite the “front” or “outside.” The terms “front,” “outside,” “back,” and “inside” have no significance other than for convenience in identifying a side of slit plane410for the description herein.

In a preferred embodiment, transducers420a . . . nmay be uniformly sized, and the size of a single transducer may be approximately 00 μm to 10 mm. Transducers of other sizes, shapes, and dimensions are well-known in the art. In general, the size and dimensions of a transducer420nfor securing to slit plane410may depend on power consumption characteristics of the transducer, efficiency of the transducer, power availability to the transducer(s), available space and dimensions of available space on slit plane410, and means for securing the transducer(s) to slit plane410. Many types of transducers are well known in the art, and may be used in this invention with necessary and well-known adaptation. Transducer types include, but are not limited to, CMUTs, piezo stacks, electrostatic, membrane, magnetostrictive, flexural, resonant cavity, and others.

The density of transducers420a . . . non slit plane410may range from as many as one transducer for every hogel down to as few as one transducer per panel. In a preferred embodiment, a rectangular slit plane410with a 1 meter diagonal may have 600×600 hogels and 600×600 ultrasonic transducers. The density of transducers420a . . . non slit plane410may be determined based on desired audio power, connectivity complexity, desired resolution of ultrasound steering and shaping, or other factors known in the art.

In a preferred embodiment, ultrasonic transducers420a . . . nmay be integrated directly onto slit plane410using integrated ultrasonic transducers. Ultrasonic transducers420a . . . nmay alternatively be secured to the slit plane by soldering or epoxy. The particular means for securing a transducer420nto slit plane410may depend at least on the material from which slit plan410is made, size of transducers, and density of transducers.

The available surface area of slit plane410, i.e., the area other than where slits430a . . . nare located, is the area where transducers420a . . . nmay be located. For example, a transducer420nplaced directly over, in whole or in part, any part of a slit430nthrough which light550nfrom 3-D display510passes may block light for 3-D display510and may thereby affect the functionality of 3-D display510. In some embodiments, because light550nfrom 3-D display510may pass through a slit at an angle, a transducer420nthat protrudes from slit plane410, which is located near a slit430n, may obstruct light550nfrom 3-D display510as the light exits the slit430n, and may thereby affect the functionality of the 3-D display.

In addition to fitting transducers420a . . . ninto the available surface area on slit plane410, transducers420a . . . nmay also be subject to minimum distances between each transducer. In one embodiment, transducers420a . . . nmay be separated by half the ultrasonic wavelength or less. This maximum separation may facilitate 180 degree acoustic operation. Other embodiments may use longer separation distances between transducers, although this may result in less than 180 degree acoustic operation. In general, the angle of possible acoustic operation may decrease as the separation distance between the transducers increases.

In another embodiment, transducers420a . . . nmay be manufactured into slit plane410. For example the front surface of slit plane410could be a material such as, or similar to, a piezo stack which could be etched and patterned to create an array of ultrasonic transducers. Another embodiment may include the formation of membranes on the front surface of slit plane410surface, such as metallic membranes formed on KOH back-etched aluminum actuated electrostatically. Other embodiments may include, but are not limited to, flexural structures, resonant cavities, and magnetostrictive structures.

Transducers420a . . . nmust be powered and controlled. In a preferred embodiment, transducers420a . . . nare powered by wires, antennae, or optical means.

Transducers420a . . . nmay be and controlled by a processor, microprocessor, microchip, or any other device or system capable of sending varied electrical signals to a transducer. For example, in a preferred embodiment, a transducer420nmay be controlled by a driving computer, or one or more audio boards synced with one or more video cards, or other means.

In an alternate embodiment, instead of being positioned on the slit plane, the transducers may be positioned on a third layer with an additional set of lenses. These lenses could be positive or negative and could increase the optical fill factor, reducing the appearance of black areas in the display, and/or allowing for further adjustment of optical scan angle in the vertical direction. By having a third plane with a set of lenslets it may be possible to have the lenslets themselves be part of an ultrasonic transducer. For example, the concave surface of a negative lenslet could be covered with a transparent conductor. A transparent, conductive membrane could be placed over this concavity to create an ultrasonic transducer. The connective traces could be located on the slit plane. In such a configuration, the layer visible to the viewer would be mostly luminous and transparent (rather than mostly opaque like the slit plane may be in some embodiments) and would allow the ultrasonic field and 3-D imagery to be fully superimposed as they are emitted from the same point on the plane. The second set of lenses would allow the first set to be made with long focal lengths, which could simplify fabrication.

FIGS.8and9illustrate this alternate embodiment.FIG.8illustrates lenslet array800, slit plane810, and a third plane820with a lenslet array and ultrasonic transducers. Lenslet801focuses input light802, which is from the 3-D display. Light focused by lenslet801then goes through slit811in slit plane810. Item803shows the focal distance of the back of lenslet array. Item804shows the focal distance on the front of the lenslet array. Distance805divided by distance806is the fill ratio. Directed acoustic field822emanates from transducer825on third plane820.

FIG.9illustrates lenslet/lenticular plane910, slit plane920, and negative lenslet array930with transparent ultrasonic membrane931. Light912from 3-D display travels through lenslet/lenticular plane910, through slit914in slit plane920, and is then modified at lenslet934. Item933shows an ultrasonic wave field from a transducer in lenslet array930. Item932shows the modified light angle resulting from lenslet array930.

3-D display510may be secured to slit plane410in many ways known in the art, including but not limited to adhesive, epoxy, air pressure, soldering, or other methods or combinations of methods.

Because slit plane410and ultrasonic transducers420a . . . nin slit plane unit500are coplanar, e.g., as illustrated inFIGS.4and5, slit plane units may be tiled to create a larger co-planar surface600as shown inFIG.6. Tiled surface600comprises, generally, multiple slit plane units500a . . . n. Tiled slit plane units500a . . . nmay be secured by mounting tiled slit plane units500a . . . nonto a common structure such as a wall or other mounting structure. The 3-D displays and sets of ultrasonic transducers respectively associated with each of tiled slit plane units500a . . . nmay be driven by any driver technologies or approaches known in the art, such as computers, or parallel networking, or hardware, or in any other manner as known in the art.

FIG.7illustrates one embodiment of a tiled surface600in a room. The individual units of tiled wall600are each a slit plane unit500, which have been tiled to cover some or all of a wall, and may be used on conjunction with one another to generate object150, which may be a 3-D virtual object having acoustic and/or tactile properties as generated by the ultrasonic transducers on the slit plane units comprising tiled wall600.

The following descriptions of three specific embodiments present three exemplary embodiments in great detail. Although a person of ordinary skill would not need such details to implement or practice the invention described herein, these descriptions of three specific exemplary embodiments are provided merely as examples of how this invention might be practiced or implemented.

In a first specific detailed embodiment, a tileable unit is comprised of a holographic video display plane, a lenticular plane and a slit plane. The holographic plane is fabricated from a 1 mm thick double-side-polished, x-cut lithium niobate wafer. The wafer is treated to possess surface waveguides. The waveguides are 800 microns wide and 48 mm long with interruptions of 1 mm every 1 mm for a 50/50 waveguide duty cycle. These waveguides are formed by proton exchange in pure benzoic acid using an aluminum mask and the waveguide extends to a depth of 0.5 microns after proton exchange. The waveguides are then annealed for 45 minutes at 375° C. Along each horizontal line defined by the intermittent waveguide, there is a column of three interdigital transducers, each corresponding to one vertical view. These transducers are patterned from the aluminum remaining from the proton exchange mask, which is then patterned to create aluminum transducers with a film thickness of 200 nm, and a transducer width of 190 microns and length of 800 microns, with a frequency chirp corresponding to a range from 300 to 600 MHz with transducer finger widths running from 2.4 to 1.6 microns.

At the other end of the proton exchange region, just before the next set of transducers, is an input coupling grating, 800 μm wide and 200 microns long etched to a depth of 150 nm into the lithium niobate. Red, green, and blue laser light (633 nm, 532 nm, and 45 nm) of TE polarization is introduced into the polished side of the polished side of the device. The device is polished at an angle of 26.565 degrees from the substrate normal. The laser light is made to enter the polished face perpendicularly to the normal so that all colors travel collinearly within the substrate and bounce on the transducer/waveguide side of the substrate at a period of 2 mm. The bounces are made to occur centered on the grating input couplers coupling light into the waveguide. RF is applied to the transducers diffracting light from the waveguide into the substrate and falling toward the bottom surface. The bottom surface is etched with a pattern of high-aspect-ratio cones (10:1 length:width) which serve as an adiabatic index shift to eliminate fresnel reflection. This pattern is modulated with a ramp at a larger spatially frequency to direct the light normally out of the bottom of the device.

The exiting light then encounters the lenticular plane. The lenticular plane is centered on the waveguide regions from the holographic video plane. The lenticular is made to have a focal length of 1 mm (in lithium niobate) and is fabricated on the bottom surface of the lithium niobate wafer. The power of the lenticular array is in the horizontal direction. The pitch of the lenticular is 1 mm. The dimensions of the lenticular are 50 mm by 50 mm. The lenticular is formed by direct-write grayscale lithography in grayscale resist. The light diffracted by the transducers is collimated in the lithium niobate substrate and then focused by the lenticular through the horizontal slit plane.

The horizontal slit plane is located 1 mm from the lenticular plane and the slit is centered on the lenticular. Each slit is 100 μm width and 1 mm length and there is one slit centered above every waveguide region. The slit is composed of a sandwich of one layer of nickel, then PZT, and then another layer of nickel. The outside nickel layers are etched to form capacitive structures which serve as ultrasonic transducers. These transducers are 900×900 microns and are separated by 2 mm, staggered between the slits so as to form a regular pattern. The smaller, thinner channels are etched to form channels which carry ultrasonic drive signal to the transducers. The nickel on the other side of these traces is removed to prevent the creation of more capacitors. The nickel faces are covered on the front by india ink to provide light absorption and, on the back, by charred photoresist which serves both to absorb light and to dampen sound. The ultrasonic transducers are driven with a carrier frequency at 40 kHz. This carrier is then amplitude modulated at lower frequencies (e.g., below 200 Hz) for tactile fields and at higher frequencies (e.g., above 200 Hz) for audio fields.

The light exiting the slit forms holographic images with full parallax and mingles with the ultrasonic wavefronts which serve to create directed parametric audio and tactile fields. The mingling of the sonic and optical fields creates the three-sense experience.

In a second specific detailed embodiment, a three-sense display unit is created from a high-resolution liquid crystal display followed by a positive lenslet array plane, an aperture plane, and finally a negative lenslet array with active membranes.

The 3-D display layer is created by placing a 400×250 mm positive lenslet with 2 mm pitch in front of a 32 inch diagonal 4 k display (3840×2160 pixels). The lenslet is a 400×250 lenslet array. The lenslet is registered to a 8×10 pixel group which defines the number of addressable views. The lenslet is separated from the display by three focal lengths. Pixels from the display are then demagnified and imaged just beyond the front focal length of the lenslet. This image forms the image plane. The aperture plane, which is the slit plane with a square or circular aperture instead of a long rectangular slit, is placed at the image plane. The size of the aperture is a circular aperture with 1 mm radius. The aperture plane contains traces for ultrasonic signals. The third layer is a negative lenslet array with the back focal length at the image plane. The negative lenslet array is covered with indium tin oxide (ITO) on the front side and connected through vias to the aperture plane traces. A buffer layer of epoxy is used to adhere a thin plastic membrane 5-10 microns thick over the top of the negative lenslet array affixed by epoxy. The membrane is also covered with indium tin oxide. The membrane is attracted and repelled by the interior of the negative lens, thereby forming an ultrasonic transducer. The light waves and ultrasonic waves combine in the far field to create a three-sense experience.

In a third specific detailed embodiment, a three-sense display unit is created from a high resolution display with a high enough frame rate to allow for use with shutter glasses such as 120 Hz (this embodiment has eyeglasses as an encumbrance). A liquid crystal display is followed by a transparent slit plane (where the slit is the entirety of the display). The transparent slit plane has transparent ITO traces on the side facing the high resolution display. The other side has electrodes on a 3 mm grid surrounded by polyimide or SU8 walls with a plastic membrane stretched and affixed over the walls to form a cavity. The top membrane is made conductive with an ITO layer forming an array of ultrasonic transducers. The visual field and ultrasonic field interact in front of the display to effect a three-sense experience.

The foregoing disclosure is presented by way of example only, and is not limiting. Various alterations, improvements, and modifications will occur and are intended to those skilled in the art, though not expressly stated herein. These alterations, improvements, and modifications are intended to be suggested hereby, and are within the spirit and scope of the invention.

The illustrations and descriptions of the invention herein have been simplified as appropriate to focus on elements essential to clearly understand the invention. Other elements may be desirable and/or required in order to implement the invention. However, because such elements are well known and do not facilitate a better understanding of the invention, a detailed discussion of such elements is not provided herein.