Abstract:
An augmented reality system with mobile and interactive functions for multiple users includes two major portions: a computer system for handling augmented reality functions, and a user system for each user. The computer system for handling augmented reality functions has very powerful functionality for processing digital image data and transforming the digital image data into a three-dimensional virtual image for each user system. The user system mainly includes a Head-Mounted Display (HMD), a microphone and a PDA. A user can see the virtual image from the HMD and use the microphone or the PDA for communication with other users.

Description:
BACKGROUND OF THE INVENTION  
       [0001]     1. Field of the Invention  
         [0002]     The present invention relates to an augmented reality system with mobile and interactive functions for multiple users that can, for example, be applied to virtual prototype evaluations for vehicles.  
         [0003]     2. Description of the Related Art  
         [0004]     Augmented reality is a new virtual reality technology that combines environmental images with computer virtual images.  
         [0005]     An augmented reality kit (ARtoolKit) can provide users with one of the most natural of browsing methods; a virtual model will move or rotate with the viewing direction of the user, which provides a more vivid experience than browsing simply with a mouse or keyboard. However, the augmented reality kit requires huge computing capabilities, which are not offered by typical mobile computing devices, such as PDAs; any computer capable of providing this huge computing ability will inevitably have a large volume and little mobility.  
         [0006]     Moreover, how to reach a discussion from different locations or even let one user can see other user&#39;s vision.  
         [0007]     Therefore, it is desirable to provide an augmented reality system with mobile and interactive functions for multiple users to mitigate and/or obviate the aforementioned problems.  
       SUMMARY OF THE INVENTION  
       [0008]     A main objective of the present invention is to provide an augmented reality system with mobile and interactive functions for multiple users that can, for example, be applied to virtual prototype evaluations for vehicles. So multiple users can have real-time discussion at different locations, and even see other user&#39;s vision. These discussions can be performed by PDA so the users can also input comment for record.  
         [0009]     In order to achieve the above-mention objective, the augmented reality system with mobile and interactive functions for multiple users includes two major portions: a computer system for handling augmented reality functions, and a user system for each user. The computer system for handling augmented reality functions has very powerful functionality for processing digital image data and transforming the digital image data into a three-dimensional virtual image for each user system.  
         [0010]     The user system includes the head-mounted display, a camera and a microphone on the display, and a portable computer. The user utilizes the head-mounted display to watch the three-dimensional virtual image and the microphone or the portable computer to communicate with other user. The camera can obtain the viewing position of the user. When the user wants to see other user&#39;s vision, the augmented reality computer system can compute the other three-dimensional virtual image watched by other user by obtaining other user&#39;s position.  
         [0011]     Other objects, advantages, and novel features of the invention will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0012]      FIG. 1  is a system structure drawing of the present invention which shows the performance environment in a single area.  
         [0013]      FIG. 2  is a system structure drawing of the present invention which shows the performance environment in two different areas.  
         [0014]      FIG. 3  is a structure drawing of a software program related to the present invention.  
         [0015]      FIG. 4  is a flow chart for displaying virtual images according to the present invention.  
         [0016]      FIG. 5  is a flow chart of showing a usage status for multiple users.  
         [0017]      FIG. 6  is a drawing of an embodiment of a portable computer according to the present invention.  
         [0018]      FIG. 7  is a schematic drawing of the present invention. 
     
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT  
       [0019]     Please refer to  FIG. 1 .  FIG. 1  is a system structure drawing of the present invention which is used for designing the appearance of vehicles.  
         [0020]     The present invention provides an augmented reality system with mobile and interactive functions for multiple users  10 , which includes two major portions: an augmented reality computer system  20 , and multiple user systems  50  for each user. In this embodiment, there are two users  80   a ,  80   b  using the augmented reality computer system at the same time.  
         [0021]     In this embodiment, the augmented reality computer system  20  comprises a first augmented reality computer subsystem  20   a  and a second augmented reality computer subsystem  20   b , wherein each subsystem  20   a ,  20   b  are basically electrically connected together. With reference also to  FIG. 3 , each subsystem  20   a ,  20   b  utilizes one computer, and each subsystem  20   a ,  20   b  comprises an augmented reality system application program  21 . In the present invention, the augmented reality system application program  21  comprises computer image generation program code  22 , data transmission program code  23 , viewing point position analysis program code  24  and three-dimensional computer drawing data  25 . In this embodiment, the three-dimensional computer drawing data  25  is related to the vehicle appearance design drawing data.  
         [0022]     The user system  50  comprises user systems  50   a ,  50   b  for each user  80   a ,  80   b . The user system  50   a  comprises a head-mounted display  30   a  (which usually includes a speaker), a camera  3  la and a microphone  32   a  mounted on the head-mounted display  30   a , and a portable computer  40   a . Similarly, the user system  50   b  also comprises a head-mounted display  30   b , a camera  31   b , a microphone  32   b  and a portable computer  40   b.    
         [0023]     In this embodiment, each user  80   a ,  80   b  wears the head-mounted display  30   a ,  30   b , and when the user  80   a ,  80   b  moves, his or her current position or the angle of his or her head changes, as does a virtual image  60  displayed a real image. There is a position reference object  70  in this embodiment; when the user  80   a ,  80   b  moves around the position reference object  70 , the virtual image  60  displayed an image would be seen at the position of the position reference object  70 , as shown in  FIG. 7 .  
         [0024]     The embodiment of  FIG. 1  is substantially a performance environment in a single area. Please refer to  FIG. 2 .  FIG. 2  is a system structure drawing of the present invention which shows a performance environment in two different areas. The subsystems  20   a ,  20   b  are electrically connected together via the Internet  90  (or via an intranet for shorter distances). Since the users  80   a ,  80   b  are located at different positions, there are two different reference objects  70   a ,  70   b , and the virtual images  60   a ,  60   b  are separately shown at the position of the reference objects  70   a ,  70   b.    
         [0025]     Please refer to  FIG. 4 .  FIG. 4  is a flow chart for displaying virtual image according to the present invention. The following description is performed at the user  80   a  end:  
         [0000]     Step  401 :  
         [0026]     The augmented reality computer subsystem  20   a  obtains the three-dimensional computer drawing data  25 .  
         [0000]     Step  402 :  
         [0027]     The image of the position reference object  70  is obtained; the camera  31   a  is placed on the head-mounted display  30   a , so when the user  80   a  faces the position reference object  70 , the camera  3  la can obtain an image of the position reference object  70  and send the image to the subsystem  20   a.    
         [0000]     Step  403 :  
         [0028]     The image of the position reference object  70  is analyzed to obtain a viewing point position parameter.  
         [0029]     The viewing point position analyze program code  24  of the subsystem  20   a  analyzes the image of position reference object  70  to obtain the position of the viewing point of the user  80   a . The position reference object  70  has a reference mark  71  (such as “MARKER”), and by analyzing the size, shape and direction of the reference mark, the position of the viewing point of the user  80   a  can be obtained, which is indicated by a viewing point position parameter (such as a coordinate or a vector, etc.). However, this is a well-known technology, and so requires no further description.  
         [0000]     Step  404 :  
         [0030]     The three-dimensional virtual image  60  is calculated according to the viewing point position parameter; with the viewing point position parameter, the computer image generation program code  22  can transform the three-dimensional computer drawing data  25  into a three-dimensional virtual image  60 . This process is a well known imaging procedure  
         [0000]     Step  405 :  
         [0031]     The virtual image  60  is sent to the head-mounted display  30  or the portable computer  40 , so that the user  80   a  can see the virtual image  60 . Please refer to  FIG. 5 .  FIG. 5  is a flow chart of showing a usage status for multiple users. The following description considers when the user  80   a  wants to send his or her comments to the user  80   b , or the user  80   b  wants to send his or her comments to the user  80   a.    
         [0000]     Step A 1 : Recording the Comment.  
         [0032]     In the present invention, the user  80   a  can record his or her comments about the virtual image  60  in the portable computer  40   a ; for example, comments about the shape or color of the vehicle, or the inputting of instructions via the portable computer  40   a  to control the subsystem  20   a  to change the shape or color of the vehicle. Please refer to  FIG. 6 . The portable computer  40  is a PDA; a screen  41  of the portable computer  40  displays a virtual image window  42  and a comment window  43 .  
         [0000]     Step A 2 : Sending the Comment.  
         [0033]     The user  80   a  sends a virtual image window  42  and a comment window  43  to the subsystem  20   b  via the subsystem  20   a  by controlling the portable computer  40   a.    
         [0000]     Step B  1 : Receiving the Comment.  
         [0034]     The subsystem  20   b  receives the virtual image window  42  and the comment window  43  sent from the subsystem  20   a  and sends the virtual image window  42  and the comment window  43  to the portable computer  40   b .  
         [0000]     Step B 2 : Executing an Image Switch Instruction.  
         [0035]     If the user  80   b  wants to have a direct discussion with the user  80   a , it is preferably to involve a discussion of the virtual image  60   a  as seen by the user  80   a . The user  80   b  can use the portable computer  40   b  to execute the image switch instruction.  
         [0000]     Step B 3 : Sending an Image Switch Execution Instruction.  
         [0036]     The subsystem  20   b  sends an image switch execution instruction to the subsystem  20   a.    
         [0000]     Step A 3 : the Subsystem  20   a  Receives the Image Switch Execution Instruction.  
         [0000]     Step A 4 : the subsystem  20   a  continuously sends the first viewing point position parameter, which is the viewing point of the user  80   a.    
         [0000]     Step B 4 : the subsystem  20   b  receives the first viewing point position parameter.  
         [0000]     Step B 5 : The three-dimensional virtual image as seen by the first user is calculated.  
         [0037]     Meanwhile, the subsystem  20   b  calculates the virtual image  60   a  according to the first viewing point position parameter.  
         [0000]     Step B 6 : The virtual image  60   a  is sent to the head-mounted display  30   b  and the portable computer  40   b.    
         [0038]     The user  80   b  can thus see on the head-mounted display  30   b  the image seen by the user  80   a . Since the first viewing point position parameter is a small sized piece of data, so it can be sent quickly.  
         [0039]     Of course, while the user  80   a  changes his or her viewing position, step A 4  will continuously be performed, as do steps B 4 -B 6 .  
         [0040]     Furthermore, the users  80   a ,  80   b  can communicate via audio, particularly when the users  80   a ,  80   b  are at different positions (as shown in  FIG. 2 ). For example, there are microphones  32   a ,  32   b  mounted in the head-mounted displays  30   a ,  30   b , and the head-mounted displays  30   a ,  30   b  also have built-in speakers for real-time communications. Therefore, there may be no comments, and consequently no steps A 1 , A 2 , B 1 . Of course, the portable computers  40   a ,  40   b  may also have built-in microphones and speakers (not shown), in which case no microphones  32   a ,  32   b  and speakers need to be mounted in the head-mounted display  30   a ,  30   b.    
         [0041]     Data transmissions between the two subsystems  20   a ,  20   b  or between the subsystems  20   a ,  20   b  and the portable computer  40   a ,  40   b  can be performed by the data transmission program code  23 .  
         [0042]     Although the present invention has been explained in relation to its preferred embodiment, it is to be understood that many other possible modifications and variations can be made without departing from the spirit and scope of the invention as hereinafter claimed. For example. The augmented reality computer system  20  shown in  FIG. 1  can be one super computer, and so there would be no need for other subsystems, or the three-dimensional computer drawing data  25  can be stored in another computer for sharing with the two computers.