Abstract:
A virtual encounter system includes a mannequin having life-like features. The mannequin includes a body, a camera, coupled to the body, for sending video signals over a communications network, and a microphone, coupled to the body, for sending audio signals to the communications network. The system also includes a set of goggles that includes a display to render video signals received from the camera and a transducer to transduce the audio signals received from the microphone.

Description:
TECHNICAL FIELD  
       [0001]     This disclosure relates to virtual reality devices, and in particular, using these devices for communication and contact.  
       BACKGROUND  
       [0002]     Two people can be separated by thousands of miles or across a town. With the development of the telephone, two people can hear each other&#39;s voice, and, to each of them, the experience is as if the other person was right next to them. Other developments have increased the perception of physical closeness. For example, teleconferencing and Internet cameras allow two people to see each other as well as hear each other over long distances.  
       SUMMARY  
       [0003]     In one aspect, the invention is a virtual encounter system includes a mannequin having life-like features. The mannequin includes a body, a camera, coupled to the body, for sending video signals over a communications network, and a microphone, coupled to the body, for sending audio signals to the communications network. The system also includes a set of goggles including a display to render the video signals received from the camera and a transducer to transduce the audio signals received from the microphone.  
         [0004]     In another aspect, the invention is a method of having a virtual encounter. The method includes sending audio signals over a communications network. The audio signals are produced from a microphone coupled to a mannequin having life-like features. The method also includes sending video signals over the communications network. The video signals are produced from a camera coupled to the mannequin. The method further includes rendering the video signals received from the communications network using a display device embedded in a set of goggles and transducing the audio signals received from the communications network using a transducer embedded in the set of goggles.  
         [0005]     One or more of the aspects above have one or more of the following advantages. The virtual encounter system adds a higher level of perception that two people are in the same place. Aspects of the system allow two people to touch and to feel each other as well as manipulate objects in each other&#39;s environment. Thus, a business person can shake a client&#39;s hand from across an ocean. Parents on business trips can read to their children at home and put them to bed. People using the system while in two different locations can interact with each other in a virtual environment of their own selection, e.g., a beach or a mountaintop. People can change their physical appearance in the virtual environment so that they seem taller or thinner to the other person or become any entity of their own choosing. 
     
    
     DESCRIPTION OF THE DRAWINGS  
       [0006]      FIG. 1  is a view of a virtual encounter system.  
         [0007]      FIG. 2A  is a view of a left side of a head of a mannequin.  
         [0008]      FIG. 2B  is a view of a right side of the head of the mannequin.  
         [0009]      FIG. 3  is a view of a set of virtual glasses.  
         [0010]      FIG. 4  is a view of a wireless earphone.  
         [0011]      FIG. 5  is a functional diagram of the virtual encounter system.  
         [0012]      FIG. 6  is a signal flow diagram of the virtual encounter system.  
         [0013]      FIG. 7A  is a view of a user with motion sensors.  
         [0014]      FIG. 7B  is a view of a robot with motion actuators.  
         [0015]      FIG. 8A  is a view of a left hand of the robot.  
         [0016]      FIG. 8B  is a view a left glove worn by the user.  
         [0017]      FIG. 9A  is a view of a robot with tactile actuators.  
         [0018]      FIG. 9B  is a view of the user with tactile sensors.  
         [0019]      FIG. 10A  is a view of a scene with the user in a room.  
         [0020]      FIG. 10B  is a view of the scene with the user on a beach.  
         [0021]      FIG. 11A  is a view of an image of the user.  
         [0022]      FIG. 11B  is a view of a morphed image of the user. 
     
    
     DESCRIPTION  
       [0023]     Referring to  FIG. 1 , a virtual encounter system  10  includes in a first location A, a mannequin  12   a , a communication gateway  16   a , a set of goggles  20   a  worn by a user  22   a , and two wireless earphones (earphone  24   a  and earphone  26   a ) also worn by user  22   a . System  10  can further include in a location B, a mannequin  12   b , a communication gateway  16   b , a set of goggles  20   b  worn by a user  22   b , and two wireless earphones (earphone  24   b  and earphone  26   b ) also worn by user  22   b . Gateway  16   a  and gateway  16   b  are connected by a network  24  (e.g., the Internet).  
         [0024]     As will be explained below, when user  22   a  interacts with mannequin  12   a  in location A by seeing and hearing the mannequin, user  22   a  perceives seeing user  22   b  and hearing user  22   b  in location B. Likewise, user  22   b  listens and sees mannequin  12   b  but perceives listening and seeing user  22   a  in location A. Details of the gateways  16   a  and  16   b  are discussed below. Suffice it to say that the gateways  16   a  and  16   b  execute processes to process and transport raw data produced for instance when users  22   a  and  22   b  interact with respective mannequins  12   a  and  12   b.    
         [0025]     Referring to  FIGS. 2A and 2B , each mannequin  12   a - 12   b  includes a camera (e.g., camera  30   a  and camera  30   b ) positioned in a left eye socket (e.g., left eye socket  34   a  and left eye socket  34   b ), and a camera (e.g., camera  36   a  and camera  36   b ) positioned in a right eye socket (e.g., right eye socket  38   a  and right eye socket  38   b ).  
         [0026]     Each mannequin  12   a - 12   b  also includes a microphone (e.g., microphone  42   a  and microphone  42   b ) positioned within a left ear (e.g., left ear  46   a  and left ear  46   b ), and a microphone (e.g., microphone  48   a  and microphone  48   b ) positioned within a right ear (e.g., right ear  52   a  and right ear  52   b ).  
         [0027]     Each mannequin  12   a - 12   b  further includes a transmitter (e.g., transmitter  72   a  and transmitter  72   b ) containing a battery (not shown). Transmitters  72   a - 72   b  send the audio and video signals from the cameras and the microphones to communication gateway  16   a - 16   b.    
         [0028]     Referring to  FIG. 3 , each set of goggles  20   a  and  20   b  includes one left display (left display  56   a  and left display  56   b ) and one right display (right display  60   a  and right display  60   b ). Each set of goggles  20   a  and  20   b  includes a receiver (e.g., receiver  70   a  and receiver  70   b ) containing a battery source (not shown). Receivers  70   a - 70   b  receive the audio and video signals transmitted from processors  16   a - 16   b.    
         [0029]     Referring to  FIG. 4 , each earphone  24   a ,  24   b ,  26   a  and  26   b  includes a receiver  74  for receiving audio signals from a corresponding microphone  42   a ,  42   b ,  48   a  and  48   b  an amplifier  75  for amplifying the audio signal and a transducer  76  for broadcasting audio signals.  
         [0030]     Referring to  FIG. 5 , each communication gateway  16   a - 16   b  includes an adapter  78   a - 78   b , a processor  80   a - 80   b , memory  84   a - 84   b , an interface  88   a - 88   b  and a storage medium  92   a - 92   b  (e.g., a hard disk). Each adapter  78   a - 78   b  establishes a bi-directional signal connection with network  24 .  
         [0031]     Each interface  88   a - 88   b  receives, via transmitter  72   a - 78   b  in mannequin  12   a - 12   b , video signals from cameras  30   a - 30   b ,  36   a - 36   b  and audio signals from microphones  42   a - 42   b ,  48   a - 48   b . Each interface  88   a - 88   b  sends video signals to displays  56   a ,  56   b  in goggles  20   a - 20   b  via receiver  70   a - 70   b . Each interface  88   a  sends audio signals to earphones  24   a - 24   b ,  26   a - 26   b  in goggles  20   a - 20   b  via receiver  74   a - 74   b.    
         [0032]     Each storage medium  92   a - 92   b  stores an operating system  96   a - 96   b , data  98   a - 98   b  for establishing communications links with other communication gateways, and computer instructions  94   a - 94   b  which are executed by processor  80   a - 80   b  in respective memories  84   a - 84   b  to coordinate, send and receive audio, visual and other sensory signals to and from network  24 .  
         [0033]     Signals within system  10  are sent using a standard streaming connection using time-stamped packets or a stream of bits over a continuous connection. Other examples, include using a direct connection such as an integrated services digital network (ISDN).  
         [0034]     Referring to  FIG. 6 , in operation, camera  30   b  and camera  36   b  record video images from Location B. The video images are transmitted wirelessly to communication gateway  16   b  as video signals. Communication gateway  16   b  sends the video signals through network  28  to communication gateway  16   a . Communication gateway  16   b  transmits the video signals wirelessly to set of goggles  20   a . The video images recorded by camera  30   b  are rendered on to display  56   a , and the video images recorded on camera  36   b  are rendered on to display  60   a.    
         [0035]     Likewise, communication gateway  16   a  and communication gateway  16   b  work in the opposite direction through network  24 , so that the video images, from location A, recorded by camera  30   a  are rendered on to display  56   b . The video images, recorded by camera  36   a  are rendered on display  60   b.    
         [0036]     The sounds received by microphone  42   a  in location A, are transmitted to earphone  24   b  and sounds received in location A by microphone  52   a  are transmitted to earphone  26   b . The sounds received by microphone  42   b  in location B, are transmitted to earphone  24   a  and sounds received in location B by microphone  52   b  are transmitted to earphone  26   a.    
         [0037]     Using system  10 , two people can have a conversation where each of the persons perceives that the other is in the same location as them.  
         [0038]     Referring to  FIGS. 7A and 7B , the user  22   a  is shown wearing motion sensors  101 , over portions of their bodies, and in particular over those portions of the body that exhibit movement. In addition, the mannequins are replaced by robots. For example, a robot  12   b  includes a series of motion actuators  103 . Each motion actuator  103  placement corresponds to a motion sensor  101  on the user  22   a  so that each motion sensor activates a motion actuator in the robot that makes the corresponding movement.  
         [0039]     For example, when the user  22   a  moves their right hand, a sensor in the right hand sends a signal through the network to a motion actuator on the robot. The robot  12   b  in turn moves its right hand.  
         [0040]     In another example, a user  22   a  can walk towards a robot  12   a  in location A. All the sensors on the user  22   a  send a corresponding signal to the actuators on the robot  12   b  in location B. The robot  12   b  in location B performs the same walking movement. The user  22   b  in location B is not looking in location B but rather through the eyes of the robot  12   a  in location A so that user  22   b  does see the user  22   a  in location A walking towards them, but not because the robot  12   b  in location B is walking. However, the fact that the robot  12   b  in location B is walking enables two things to happen. First, since the user  22   a  in location A is seeing through the eyes of the robot  12   b  in location B and since the robot  12   b  in location B is walking enables the user  22   a  in location A to see what he would see if he were indeed walking in location B. Second, it enables the robot  12   b  in location B to meet up with the user  22   b  in location B.  
         [0041]     Referring to  FIGS. 8A and 8B , in still other embodiments, tactile sensors  104  are placed on the exterior of a robot hand  102  located in Location A. Corresponding tactile actuators  106  are sewn into an interior of a glove  104  worn by a user in location B. Using system  10 , a user in location B can feel objects in Location A. For example, a user can see a vase within a room, walk over to the vase, and pick-up the vase. The tactile sensors-actuators are sensitive enough so that the user can feel the texture of the vase.  
         [0042]     Referring to  FIGS. 9A and 9B , in other embodiments, sensors are placed over various parts of a robot. Corresponding actuators can be sewn in the interior of a body suit that is worn by a user. The sensors and their corresponding actuators are calibrated so that more sensitive regions of a human are calibrated with a higher degree of sensitivity.  
         [0043]     Referring to  FIGS. 10A and 10B  in other embodiments, user  22   a  can receive an image of a user  22   b  but the actual background behind user  22   b  is altered. For example, user  22   b  is in a room  202  but user  22   a  perceives user  22   b  on a beach  206  or on a mountaintop (not shown). Using conventional video image editing techniques, the communication gateway  16   a  processes the signals received from Location B and removes or blanks-out the video image except for the portion that has the user  22   b . For the blanked out areas on the image, the communication gateway  16   a  overlays a replacement background, e.g., virtual environment to have the user  22   b  appear to user  22   a  in a different environment. Generally, the system can be configured so that either user  22   a  or user  22   b  can control how the user  22   b  is perceived by the user  22   a . Communication gateway  16   a  using conventional techniques can supplement the audio signals received with stored virtual sounds. For example, waves are added to a beach scene, or eagles screaming are added to a mountaintop scene.  
         [0044]     In addition, gateway  16   a  can also supplement tactile sensations with stored virtual tactile sensations. For example, a user can feel the sand on her feet in the beach scene or a cold breeze on her cheeks in a mountain top scene.  
         [0045]     In this embodiment, each storage medium  92   a - 92   b  stores data  98   a - 98   b  for generating a virtual environment including virtual visual images, virtual audio signals, and virtual tactile signals. Computer instructions  94   a - 94   b , which are executed by processor  80   a - 80   b  out of memory  84   a - 84   b , combine the visual, audio, and tactile signals received with the stored virtual visual, virtual audio and virtual tactile signals in data  98   a - 98   b.    
         [0046]     Referring to  FIGS. 11A and 11B , in other embodiments, a user  22   a  can receive a morphed image  304  of user  22   b . For example, an image  302  of user  22   b  is transmitted through network  24  to communications gateway  16   a . User  22   b  has brown hair, brown eyes and a large nose. Communications gateway  16   a  again using conventional imaging morphing techniques alters the image of user  22   b  so that user  22   b  has blond hair, blue eyes and a small noise and sends that image to goggles  20   a  to be rendered.  
         [0047]     Communication gateway  16   a  also changes the sound user  22   b  makes as perceived by user  22   a . For example, user  22   b  has a high-pitched squeaky voice. Communication gateway  22   b  using conventional techniques can alter the audio signal representing the voice of user  22   b  to be a low deep voice.  
         [0048]     In addition, communication gateway  16   a  can alter the tactile sensation. For example, user  22   b  has cold, dry and scaling skin. Communications gateway  16   a  can alter the perception of user  22   a  by sending tactile signals that make the skin of user  22   b  seem smooth and soft.  
         [0049]     In this embodiment, each storage medium  92   a - 92   b  stores data  98   a - 98   b  for generating a morph personality. Computer instructions  94   a - 94   b , which are executed by processor  80   a - 80   b  out of memory  84   a - 84   b , combine the visual, audio, and tactile signals received with the stored virtual visual, virtual audio and virtual tactile signals of a personality in data  98   a - 98   b.    
         [0050]     Thus using system  10  anyone can assume any other identity if it is stored in data  98   a - 98   b.    
         [0051]     In other embodiments, earphones are connected to the goggles. The goggles and the earphones are hooked by a cable to a port (not shown) on the communication gateway.  
         [0052]     Other embodiments not described herein are also within the scope of the following claims.