Method and apparatus for presentation of on-line directional sound

Abstract: There is disclosed an omnipresent sound system for use by a listener in an artificial reality system which operates to couple sound with the presented objects such that as the sound moves with respect to the user, the user will have the perception of the changing sound both in pitch and in volume. The sound system (10) is comprised of a series of piezoelectric elements (102) spaced apart around a user's head. The system is designed to program each element individually so as to create the illusion of omnipresent three-dimensional sound in conjunction with images presented to the listener, which images define an artificial environment.

Description: TECHNICAL FIELD OF THE INVENTION This invention relates to sound systems and more particularly to such systems that present sound to a listener in a "full wrap-around" manner. REFERENCE TO RELATED APPLICATIONS The following patent applications are referred to, and both have been assigned to Texas Instruments Incorporated. These applications have been concurrently filed and are hereby incorporated in this patent application by reference. Serial Number Title 07/593,190 Method and Apparatus for Providing a Portable Visual Display (=EP-A-0 479 605) 07/593,823 Method and Apparatus for Monitoring Physical Positioning of a User (=EP-A-0 479 603) BACKGROUND OF THE INVENTION There are many situations in which full wrap-around sound is desirable for presentation to a listener. One such system is in artificial reality systems where an artificial environment has been created for a user. This environment would typically contain scenes that are created surrounding the viewer, and as the viewer moves in any plane relative to the environment, the scene changes as it would if the viewer were moving in a real environment. In some situations sound is presented to the viewer in conjunction with the projected images such that as the viewer (or sound source) moves closer or further away, the sound will increase or decrease in volume and frequency. At times the sound would appear to come from directly behind, over or below the listener. Conventional stereophonic speakers which are placed over each ear can only approximate the direction of the sound from one side or the other. These speakers are expensive and do not satisfy the requirement that the listener actually hear the sound coming from behind, above, below or from one side. Currently, surround sound or holographic sound or what is called 3-D sound, is generated by using powerful digital signal processors which are basically stereo earphones tricking the ear into perceiving that the sound is coming from behind, around, or from different distances. Thus, a need exists in the art for a speaker system which is lightweight, inexpensive and yet which can create the illusion of sound coming from an omnipresent direction. There is a further need in the art for such a system which can be created and controlled by an artificial reality system and is portable and generatable under processor control. US-A-4 952 024 discloses a three-dimensional sight and sound reproduction apparatus including a headset having a central rib connecting an eye-covering portion. The eye covering portion includes first and second liquid crystal display screens spaced apart sideways. Depending ear limbs are also provided containing first and second earphones adjacent to the ears of the wearer. A respective signal source is provided for each earphone. US-A-4 751 419 discloses a piezoelectric oscillation assembly in which several piezoelectric oscillation devices are included, each device including a first oscillation plate, a piezoelectric oscillation element attached to the first oscillation plate, and a second oscillation plate layered against the first oscillation plate so as to define an acoustically enclosed space between them. At least one of the oscillation plates is common to all of the several piezoelectric oscillation devices. GB-A-2 039 468 discloses a head-coupled area-of-interest visual display apparatus particularly for ground-based craft-flight simulators. An image generator of the computer generated image type is used and a pilot wearing a helmet is seated within a part-spherical shell having a retro-reflective interior surface. Sensing means detect head/helmet movement to permit voluntary scanning of a wide angle of simulated view from the craft. The present invention provides a sound system for use in creating the illusion of holographic sound in an artificial reality system, said sound system comprising: a headband to be worn by a user; characterized in that a plurality of discrete vibratory elements are placed circumferentially around said headband, each of said elements being arranged for selectively receiving a respective individual electrical impulse signal and for producing sound waves in response thereto. The present invention also provides a method of creating a sound system including placing a headband around the head of a user; characterized by: distributing a plurality of discrete vibratory elements circumferentially around said headband, each of said elements being arranged for selectively receiving a respective individual electrical impulse signal and for producing sound waves in response thereto. An omnipresent sound system shown in the accompanying drawings uses a series of discreet piezoelectric elements in a headband around the user's head. Each element is individually controllable from signals provided to it from the processor. Thus, sound can be programmed to occur at any position around the user's head. By judiciously selecting the volume and timing of the sounds from each of the individual piezoelectric elements, the listener will have the sensation of omnipresent stereographic sound which is keyed to the scene being viewed by the viewer in the artificial reality system. The scene being viewed is, in turn, controllable by the sensed motion of the user. Thus, for example, a user can be walking (in the artificially created environment) toward a siren (for example, a flashing light on a vehicle), and the scene of the siren source (vehicle) would be getting larger. Concurrently, the sound from the siren would become louder, and the frequency of the siren could also be changing depending upon the approach angle. Of course, the vehicle could be moving relative to the viewer and the sound would change accordingly. The system to create this illusion of changing sound is designed to send signals to different ones of the piezoelectric elements from time to time to create the illusion of the sound moving. These sound attributes of the created environment are stored in a portable processor and provided to the viewer in conjunction with the proper scene. Thus, the system knows how far the viewer is from an object so it, in turn, knows which one of the piezoelectric elements to turn on. The system knows that as the viewer approaches to view objects, the sound should get louder. If the viewer is looking directly at an object, the sound should come from in front. The viewed objects can also have boundary boxes around them so if the viewer moves into the object, a bump can be heard. In addition, the piezoelectric devices can be made to physically vibrate, thereby creating the "feeling" of sound impacting the viewer. Since the system is object oriented, the closer the viewer gets to an object, the more the sound will change. This is important for applications, such as military and flight simulations. Thus, it is a technical advantage of the illustrated embodiment of the invention to provide a sound system which is omnipresent with respect to a viewer and which is lightweight and inexpensive to manufacture. It is a further technical advantage of the illustrated embodiment that such a system is designed to respond to signals provided by a portable processor and generated in conjunction with an artificial reality system to provide sound in coordination with the derived physical presentation of the environment around the viewer. Ways of carrying out the invention will now be described by way of example only with reference to the accompanying drawing which show one embodiment of the invention and in which like reference numbers indicate like features wherein: FIGURE 1 is a helmet mounted virtual reality device with the speaker elements exposed; FIGURE 2 is also a helmet mounted virtual reality device; FIGURE 3 is a schematic representation of a simulated reality system; FIGURES 4a, 4b and 4c depict a CCD position-orientation sensor in various orientations; and FIGURE 5 shows a three-dimensional position-orientation sensor. FIGURE 1 depicts a portable virtual reality system 10 worn by an individual on his or her head. System 10 consists of a color liquid display screen 101, an array of piezoelectric film elements 102, a position-orientation sensor 12 and a processor 13. Processor 13 generates a visual picture according to helmet 10 orientation information from position-orientation sensor 12 and on board software. Processor 13 creates a three dimensional environment and projects a view of it on screen 101. As the user moves his/her head and, hence, helmet 10, processor 13 changes the image on screen 13 to mimic the view the user would perceive if he/she were actually in the three dimensional environment. Similarly, if the user walks or runs to a new location, processor 13 changes the image on screen 101 as if the user walked or ran the same distance and direction in the three dimensional environment. Note that while screen 101 is a color liquid crystal display, it can be any type of display and can, for example, be positioned close to a user's eyes with a short focal length. Processor 13 also generates a sound field through piezoelectric elements 102 of sound band 11. Individual elements 102 are separately driven by processor 13. The processor selectively powers piezoelectric film elements 102 on an individual basis to create a directional sound field. By doing so, the processor can create the illusion of a moving sound source and of a stationary sound source when the user's head or body moves. The sound source would be stationary, i.e., the same sound would continue to come from the same elements when the user stops moving. The sound elements can be small chips or elongated bands, each driven by a separate signal from the processor. FIGURE 2 shows a system in which the user, using control 21, manually changes the presented image or manually creates a change in direction or speed with respect to the created environment. Band 22 contains elements 102 and can be adjusted via tightening mechanism 20, which can also be a volume control for the elements. FIGURE 3 schematically depicts processor 13, screen 101, speaker elements 11, joystick 21, position-orientation sensor 12 and direction control module 306. Processor 13 takes graphic information stored in a data base and generates images that are displayed on screen 101. Processor 13 also generates sound projected from piezoelectric film segments 102. Processor 13 could be a single processor or multiple processors such as a graphics processor from the TMS340 series and a digital signal processor from the TMS320 series, all available from Texas Instruments Incorporated. The '340 generates images shown on screen 101 and the '320 generates sound on element band 11. Connected to processor 13 is a position-orientation sensor 12. Position-orientation sensor 12 senses the direction that the user is looking. A flux gate compass (not shown) may also be linked to processor 13 to provide absolute north-south orientation information. Direction control block 306 provides processor 13 with information indicating the user's location and view within the three dimensional environment. Direction control block 306 receives user orientation information from position sensor 12 through processor 13 and from the user directly through joystick 21. Direction control block 306 can determine the user's position within the three dimensional environment by mathematically integrating the instantaneous orientation information from position-orientation sensor 12. FIGURES 4a, b and c depict an element in an embodiment of a position and orientation sensor 12. Container 40 is fluid filled and has light source 43 (or other source of electronic signals such as infrared or microwave) on one side and CCD 44 ("charge coupled device") or other electronic signal detectors on the other. CCD 44 is able to sense where light 401 impinges on it from source 43 and when light is blocked by fluid 42. FIGURES 4b and 4c depict different orientations of assembly 40 and hence depict different levels of fluid in device 40. In FIGURE 4b as assembly 40 is tilted down, a larger area of CCD 44 is blocked by fluid 42, allowing less light 401 to strike CCD 44. The amount of light impacting CCD 44 can be detected, for example, by using an array of individual CCD (or other detectors) devices, and monitoring, perhaps on a digital basis, the light level. When horizontal, no light gets through to CCD 44. In FIGURE 4c fluid completely covers CCD 44. In FIGURE 5 a number of CCD assemblies 40 can be combined to indicate the tilt in different axes. Three assemblies, 40a, 40b and 40c are aligned along mutually orthogonal axes and encapsulated in direction sensor 12 to detect motion in three dimensions. The orientation information from assemblies 40a, 40b and 40c is transmitted through cable 301 to processor 13. The CCD assemblies can also provide information on the motion and position of sensor 12 by examining the output of CCD device over time. It is possible to use only two devices to obtain all three orientations by taking into account the slope of liquid within each device 40. Although this description describes the invention with reference to the above specified embodiments, it is but one example, and the claims, not this description, limit the scope of the invention. Various modifications of the disclosed embodiment, as well as alternative embodiments of the invention, will become apparent to persons skilled in the art upon reference to the above description. Therefore, the appended claims will cover such modifications that fall within the true scope of the invention.

Claims: A sound system for use in creating the illusion of holographic sound in an artificial reality system, said sound system comprising: a headband (11) to be worn by a user; characterized in that a plurality of discrete vibratory elements (102) are placed circumferentially around said headband (11), each of said elements (102) being arranged for selectively receiving a respective individual electrical impulse signal and for producing sound waves in response thereto. The system set forth in claim 1, wherein said vibratory elements (102) are piezoelectric strips. The system set forth in claim 1, wherein said sound system further comprises: a processor (13) for coordinating said impulse signals to said elements (102). The system set forth in claim 1, wherein said artificial reality system is operable for creating artificial viewing environments for a plurality of users, each of said environments including predetermined structures, said artificial reality system comprising: a geographical sensor (12) connectable to each of said users for providing data pertaining to the relative movement of said user; a processor (13) local to each of said users for accepting said data from said local user and from a plurality of other users; and a presentation media (101) for providing views of said created environment to each of said users, said views being controlled by said local processor and in accordance with said accepted data from said local user and data from said other users; and wherein said processor is further operative for coordinating the provision of said impulse signals to said elements (102) in association with said provided views to said user. The system set forth in claim 4,wherein said artificial environment for each of said users is created by said local processor and wherein the environment and said impulse signals for each of said users is unique to said user. The system set forth in claim 1, wherein said artificial reality system is operable for creating an artificial viewing environment for a user, said environment including predetermined structures, said artificial reality system comprising: a geographical sensor (12) connectable to said user for providing data pertaining to the relative movement of said user; a processor (13) local to said user for accepting said data from said local user; a presentation media (101) for providing views of said created environment to said user, said views being controlled by said local processor and dependent upon said accepted data from said local user; and wherein said processor is further operative for coordinating said impulse signals to said elements in association with said provided views to said user. A method of creating a sound system including placing a headband (11) around the head of a user; characterized by: distributing a plurality of discrete vibratory elements (102) circumferentially around said headband, each of said elements (102) being arranged for selectively receiving a respective individual electrical impulse signal and for producing sound waves in response thereto. The method set forth in claim 7, further comprising: providing electrical impulses to selected discrete elements in accordance with the direction of origin from which a particular sound is to be perceived by said user. The method set forth in claim 8, wherein said vibratory elements (102) are piezoelectric elements. The method set forth in claim 7, wherein said method further comprises coordinating via a processor (13) integral with said headband (11), the provision of said impulse signals to said elements (102). The method set forth in claim 7, further comprising: associating said sound with an artificial reality system which is operable for creating artificial viewing environments for a plurality of users, each of said environments including predetermined structures. The method set forth in claim 11, further comprising: connecting a geographical sensor (12) to each said user for providing data pertaining to the relative movement of said user; accepting, via a processor (13) local to each of said viewers, said data from said local user and from a plurality of other users; and providing views of said created environment to each of said users, said views being controlled by said local processor (13) and dependent upon said accepted data from said local user and data from said other viewers. The method set forth in claim 11, wherein said artificial environment for each of said users is created by said local processor (13) and wherein the environment and said impulse signals for each of said users is unique to said user.

Inventors: CONE DALE A, JOSEPHSON NORMAN A, LEWIS RUSSELL F, MARTIN STEVEN F, CONE, DALE A., JOSEPHSON, NORMAN A., LEWIS, RUSSELL F., MARTIN, STEVEN F.
Applicants: TEXAS INSTRUMENTS INC, TEXAS INSTRUMENTS INCORPORATED
Application year: 1991
Main classification: H04R3
IPC codes: G09B9, H04R1, H04S1, H04R17, H04R3, H04R5