Patent Publication Number: US-7588516-B2

Title: Virtual reality system locomotion interface utilizing a pressure-sensing mat

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
   This is a Divisional of application Ser. No. 10/417,095, filed Apr. 17, 2003, which is a Continuation-in-Part-of application Ser. No. 10/045,052, filed Jan. 15, 2002. The disclosures of the prior applications are hereby incorporated by reference herein in their entirety. 

   BACKGROUND OF THE INVENTION 
   1. Field of Invention 
   This invention relates to virtual reality systems that can be used to fully immerse a user in virtual space. 
   2. Description of Related Art 
   Virtual reality is a computer-generated environment in which a user is immersed. Actions of the user are translated by a computer into inputs that effect the virtual environment (VE). Virtual reality systems may stimulate naturally occurring senses, such as sight, sound, touch and movement, so that a user can navigate through a virtual environment as if in the real world. 
   A major challenge to virtual reality system designers is to design a virtual reality system that allows natural human locomotion. Previous virtual reality systems that allow the user to move naturally require complex and expensive equipment. Other virtual reality systems abandon the concept of natural human locomotion, using simple hardware that allow the user to navigate through the virtual environment with artificial gestures, such as flying in the virtual space in the direction the user&#39;s finger is pointing. 
   Known virtual reality systems include treadmill devices that track the user&#39;s movement on the treadmill. Such a device is disclosed in U.S. Pat. No. 5,562,572 to Carmein. Although these treadmill devices allow movement in the user&#39;s upright position, they do not allow movement in the user&#39;s prone position. They also cannot sense whether the user is in the standing, crawling or prone position. Further, these treadmill devices are often mechanically complicated, and are thus encumbered by the inherent lag times and momentum problems associated with moving mechanical masses. 
   Other known virtual reality systems allow the user to move in the prone position, but sacrifice natural motion. For example, one known device includes a simple foot-pedal interface, similar to the accelerator of an automobile. The foot-pedal allows the user to move forward or backward, depending on where the user presses the foot-pedal. In this system, the user always moves toward the center of the field of view, and the field of view is rotated if the user turns his head past a certain angle. Although this system allows a user to navigate from any posture, the user must be in constant contact with the foot-pedal to navigate. It also does not enable the user to move naturally. 
   SUMMARY OF THE INVENTION 
   In various exemplary embodiments, the virtual reality system according to one aspect of this invention includes a pressure-sensing mat that outputs signals indicative of a user&#39;s position in real space. A virtual reality processor uses the signals output by the pressure-sensing mat to produce an output indicative of the virtual space corresponding to the user&#39;s position and movement in real space. A display device uses the output from the virtual reality processor to allow the user to be fully immersed in the virtual space. 
   In various exemplary embodiments, the pressure sensing mat includes a base layer, a plurality of pressure sensing elements formed over the base layer, and a top layer formed over the plurality of pressure-sensing elements. The plurality of pressure sensing elements output a signal indicative of pressure applied to the top layer. 
   This invention provides a virtual reality system that has a simple design and that allows a user to move naturally in any direction from any posture (e.g., standing, crawling, prone). The virtual reality system according to this invention has many advantages over previous virtual reality systems. The enhanced flexibility of the various exemplary embodiments of the system according to this invention allows a user to move forward, backward, or sideways from a prone, crawling or standing position. Thus, the virtual reality system according to this invention has many applications, such as, for example, enhanced military training, realistic video game environments, and a broad range of medical and therapeutic applications. 
   These and other features and advantages of this invention are described in, or are apparent from, the following detailed description of various exemplary embodiments of the systems and methods according to this invention. 

   
     BRIEF DESCRIPTION OF THE DRAWINGS 
     Various exemplary embodiments of this invention will be described in detail, with reference to the following figures, wherein: 
       FIG. 1  illustrates one exemplary embodiment of a virtual reality system according to this invention; 
       FIG. 2  illustrates one exemplary embodiment of the pressure sensing mat according to this invention; 
       FIG. 3  shows one exemplary embodiment of a pressure sensitive resistor usable with the various exemplary embodiments of the virtual reality system according to this invention; 
       FIG. 4  illustrates the equivalent circuit of the pressure sensing mat according to this invention; 
       FIG. 5  is a block diagram of an exemplary embodiment of the virtual reality processor according to this invention; and 
       FIG. 6  illustrates a cross section of an exemplary embodiment of a locomotion interface according to this invention. 
   

   DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
     FIG. 1  illustrates one exemplary embodiment of a virtual reality system according to this invention. The virtual reality system  1  includes a pressure sensing mat  100 , a virtual reality (VR) processor  200 , and a display  400 . It should be appreciated that the various exemplary embodiments of the virtual reality system according to this invention can have any number and configuration of components that use a pressure sensing mat to sense the user&#39;s movement in order to generate a virtual environment. 
     FIG. 2  illustrates one exemplary embodiment of the pressure sensing mat  100  according to this invention. The pressure sensing mat  100  includes a semi-rigid base layer  120 . Any suitable material can be used for the base layer  120 , such as, for example, plastic, hardwood, and polycarbonate (lexan). A grid  140  (i.e., a two-dimensional array) of pressure sensing elements  150 - 1  to  150 -n is formed over the base layer  120 . A top layer  160  is formed over the grid  140 . Any suitable layer can be used for the top layer  160 , such as, for example, rubber, natural rubber, buna&#39;s rubber, and fabric reinforced negro rubber, is preferred. 
   The pressure sensing elements  150 - 1  to  150 -n of the grid  140  detect the pressure applied to fixed points on the top layer  160  of the pressure sensing mat  100 . Any suitable pressure sensing device can be used for the pressure sensing elements  150 - 1  to  150 -n, such as, for example, electromechanical pressure sensors. In general, any known or later discovered pressure sensing device can be used for the pressure sensing elements  150 - 1  to  150 -n. 
   In the exemplary embodiment shown in  FIG. 1 , the pressure sensing elements  150 - 1  to  150 -n include force sensitive resistors. As is known in the art, force sensitive resistors include elements that act as simple voltage dividers.  FIG. 3  shows one exemplary embodiment of a pressure sensitive resistor  180  usable with the various exemplary embodiments of the virtual reality system according to this invention. The pressure sensing elements  150 - 1  to  150 -n include corresponding pressure sensitive resistors  180 - 1  to  180 -n. Each pressure sensitive resistor  180  includes an upper film  181 , a lower film  182 , a first electrode pattern  183  formed over the lower film  182 , a second electrode pattern  184  formed over the upper film  181  so as to oppose the electrode pattern  183 , and a pressure-sensitive conductor  185  formed over the second electrode pattern  184 . When the upper film  181  is pressed, the pressure sensitive conductor  185  is compressed between the first and second electrode patterns. As is known in the art, the resistance of the pressure sensitive conductor  185  is lowered when compressed. Accordingly, voltage output of the pressure sensitive resistor  180  will vary with applied pressure. For more details of a pressure-sensitive resistor, see U.S. Pat. No. 5,948,990, the disclosure of which is incorporated herein by reference. 
     FIG. 4  illustrates the equivalent circuit of the pressure sensing mat  100 . The voltage outputs Vout- 1  to Vout-n correspond to respective pressure sensing elements  150 - 1  to  150 -n that make up the grid  140 . A user applies pressure to points on the pressure sensing mat  100  as the user navigates through the virtual reality environment. The applied pressures alter the resistance of the pressure sensitive resistors  180 - 1  to  180 -n, and thus the voltage output of each of the corresponding pressure sensing elements  150 - 1  to  150 -n varies as the user moves. The grid  140  produces a voltage output that can be analyzed to generate a pattern that shadows the user&#39;s movements in the virtual space. 
     FIG. 5  is a block diagram of an exemplary embodiment of the virtual reality processor  200 . The virtual reality processor includes a controller  210 , a memory  220  (including RAM and ROM, for example), a pattern generation device  230 , a motion identification device  240 , a virtual environment rendering device  250 , an input interface  260 , and an output interface  270 . The controller  210  interfaces with the other components  220 - 270  using a control/data bus  280 . Although the exemplary virtual reality processor  200  uses a bussed architecture, it should be appreciated that the exemplary virtual reality processor  200  can use any known or later developed architectures, including ASIC, a programmed general purpose computer, discrete logic devices, etc. 
   Under control of the controller  210 , the input interface  260  can receive analog voltage signals from the pressure sensing elements  150 - 1  to  150 -n. The input interface  260  can include an analog to digital converter that converts the analog voltage signals to digital signals. The input interface  260  can input the digital signals to the memory  220  for storage. 
   Next, the controller  210  can provide the digital signals stored in the memory  220  to the pattern generation device  230 . The pattern generation device  230  samples the digital signals stored in the memory  220  at regular intervals and generates a pattern based on the digital signals at the regular intervals. The patterns generated by the pattern generation device  230  represent various positions of the user on the pressure sensing mat  100 . 
   The controller  210  transfers the patterns generated by the pattern generation device  230  to the motion identification device  240 . The motion identification device  240  can include a pattern recognition device (not shown) that identifies a given pattern with a corresponding position of the user. The pattern recognition device can identify a pattern by comparing the pattern with a database of patterns stored in the memory  220 . The pattern recognition device can also recognize the pattern based on the size, shape and/or pressure distribution of the pattern. For example, if the pattern is larger than a predetermined threshold size, the pattern recognition device will recognize the pattern as a “prone user position” pattern. Similarly, if the mat outputs signals indicative of two patterns of a similar size that alternately move, the processor determines that the user is upright (e.g., walking, running or standing (if the two patterns do not move)). If more than two smaller moving patterns are detected, the user is determined to be crawling. The patterns stored in the memory  220  can provide examples for a neural network to learn how to identify different patterns. 
   Based on the posture and directional information determined by the processor, the virtual environment (i.e., the displaying image) is appropriately altered. 
   A series of user positions identified by the pattern recognition device can be stored in the memory  220  during fixed intervals as the user navigates through the virtual environment. Preferably, the centroid of each of the patterns in the series of patterns is tracked as the user moves on the pressure sensing mat  100 . The motion identification device  240  can sample the series of user positions at the end of the fixed intervals and identify the motion of the user during the fixed intervals based on the series of user positions. The motion includes, for example, direction (forward, backward, left, right, etc.) and speed. The patterns also can be analyzed to determine the posture (standing, crawling, prone) of the user. 
   The direction that the user is facing is determined by a sensor that can be directly attached to the user. In embodiments, the sensor can be a magnetic tracker attached to the user&#39;s waist that determines the direction the waist is facing. The virtual reality system according to this invention provides significant advantages over known virtual reality systems in that only a single sensor needs to be directly attached to the user. Thus, the user is relatively free from cumbersome sensor wiring and devices. 
   The controller  210  can transpose the motion of the user into the virtual environment generated by the virtual environment rendering device  250 . Data for the virtual environment, including virtual objects, can be stored in the memory  220 . The virtual environment rendering device  250  can update the virtual environment at given intervals based on the data stored in the memory  220 . The virtual environment rendering device  250  can update the virtual space each time the user&#39;s motion is identified. Thus, as the user moves through the virtual space, the user can effect, and can be effected by, the virtual environment. For example, as the user navigates through the virtual space, the user&#39;s perspective in the virtual space can change, virtual objects can enter the user&#39;s path, and the user can move virtual objects. 
   The controller  210  can control the output interface  270  to output virtual reality environment data to the display  400 . Although the display  400  is shown in  FIG. 1  as a head-mounted display, any known or later discovered display can be used. Preferably, the display provides the user with the ability to see, hear, smell and/or touch in the virtual world so that the user is fully immersed in the virtual space. 
   In embodiments, it is conceivable that the pressure sensing mat  100  can be as large as required to allow the user to move as if the user was in the virtual space. For example, the pressure sensing mat  100  can be made to cover the floor of a large field or room. Alternatively, if space is limited, the pressure sensing mat  100  can be made smaller, in which case the user would be required to move in a bounded area or move “in place”. Further, the pressure sensing mat  100  can be arranged in a belt-like manner, such as in the form of a tread-mill. For example,  FIG. 6  illustrates a cross section of a locomotion interface  300  according to an embodiment of this invention including a pressure sensing mat  305  wrapped around a spheroid base  330 . As shown in  FIG. 6 , the pressure sensing mat  305  includes a base layer  310 , a grid  315  of pressure sensing elements  320 - 1  to  320 -n formed over the base layer  310 , and a top layer  325  formed over the grid  315 . The pressure sensing mat  305  can be held on to the surface of the spheroid base  330  by its own elasticity, thus making the contact between the spheroid base  330  and the pressure sensing mat  305  relatively frictionless. 
   The locomotion interface  300  includes a housing  335  that retains the pressure sensing mat  305  and the spheroid base  330 . Passive casters  335  can be mounted in the housing  335  to allow the pressure sensing mat  305  to move freely in all directions in the housing  335 . 
   In this embodiment, the locomotion interface  300  must have the capability of allowing a user to feel as if he/she can move in all directions for an “infinite” distance. Thus, the pressure sensing mat  305  must be able to move underneath the user as the user “moves” in the virtual environment. However, although the pressure sensing mat  305  moves will little friction on the spheroid base  330 , the mass of the pressure sensing mat  305  will not allow the user to propel the pressure sensing mat  305  underneath him/herself as the user moves. Thus, in this embodiment, the pressure sensing mat  305  is mechanically actuated to move underneath a user. For example, as shown in  FIG. 6 , a steerable roller  340  is disposed in the housing  335 , and the steerable roller is in frictional contact with the pressure sensing mat  305 . The roller  340  is steerable about a first axis  350  and a second axis  360 . The first axis  350  is perpendicular to the second axis  360 . A first motor  345  powers the roller  340  about the first axis  350 , and a second motor  355  powers the roller  340  about the second axis  360 . The thrust vectors generated by the roller  340  cause the pressure sensing mat  305  to slide around the spheroid base  330  in all directions. 
   A sensor (not shown) can be placed above the locomotion interface  300  to sense the direction in which the user is facing. The sensed user direction can then be used to determine the appropriate thrust vector to be generated by the roller  340 , to thereby move the pressure sensing mat  305  underneath the user. For example, if the sensor determines that the user is facing a first direction, the roller  340  can be controlled to create a thrust vector in the first direction to thereby move the pressure sensing mat underneath the user in a second direction opposite the first direction. 
   The roller  340 , casters  335  and other mechanical parts of the locomotion interface  300  may interfere with accurate pressure sensing by the pressure sensing mat  305 . Further, the pressure sensing mat  305  will be moving underneath the user as the user “moves” in the virtual environment, which will also diminish the accuracy of the pressure sensing. In order to solve these problems, when the user first steps on the pressure sensing mat  305 , an initialization procedure can be done in which the users position on the pressure sensing mat  305  is determined. 
   The virtual reality system  1  can be implemented as software executing on a programmed general purpose computer, a special purpose computer, a microprocessor or the like. 
   While the invention has been described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the preferred, exemplary embodiments or constructions. To the contrary, the invention is intended to cover various modifications and equivalent arrangements. In addition, while the various elements of the exemplary embodiments are shown in various combinations and configurations, which are exemplary, other combinations and configurations, including more, less or only a single element, are also within the spirit and scope of the invention.