Patent Publication Number: US-9843772-B2

Title: Method for providing telepresence using avatars, and system and computer-readable recording medium using the same

Description:
CROSS REFERENCE TO RELATED APPLICATION 
     This application claims priority to and incorporates herein by reference all disclosure in Korean Patent Application No. 10-2015-0003706 filed Jan. 9, 2015. 
     FIELD OF THE INVENTION 
     The present invention relates to a method for providing telepresence by using avatars and a system and a computer-readable recording media using the same; and more particularly, to a technology for changing a motion adaptively by recognizing information on space and considering information on a location, an orientation, etc. of an object in the space to implement an effect of enabling a first person in a first space and a second person in a second space to gather in the first space through an avatar of the second person and/or gather in the second space through an avatar of the first person. 
     BACKGROUND OF THE INVENTION 
     A telepresence system by using avatars may be applied to a technology for allowing participants at distant places to feel as if they were at one place. 
     Accordingly, the technology allows people who are far away from each other to feel as if they were together at his or her present space by using avatars. 
     Therefore, technologies relating to virtual video calls by using avatars have been disclosed in Korean Laid-Open Publication No. 2005-7017989, U.S. Pat. No. 5,736,982 and United States Laid-Open Publication No. 2008-0082311. 
     According to the Korean Laid-Open Publication No. 2005-7017989, a technology for allowing a user to control a pose of his or her avatar easily by using a keyboard, mouse, facial expression, motion, etc. is disclosed. Besides, according to the U.S. Pat. No. 5,736,982, a technology for identifying an intention of a user who accesses a virtual world to talk with another avatar (user) by using information on the user&#39;s gaze and his or her avatar&#39;s location, and then connecting a speech channel between the two users is disclosed. According to the U.S. Laid-Open Publication No. 2008-0082311, a technology for allowing a user to invite a person at a remote place in a present virtual space and play a computer game or watch a TV together with the person is disclosed. 
     Accordingly, the conventional technologies can control avatars and mutually connecting them to each other but they have a problem in that they do not control their motions adaptively in an environment where the users can co-exist at their own spaces and take their poses naturally. 
     Therefore, the inventor came to develop a technology for controlling a motion of a user&#39;s avatar adaptively even in another person&#39;s virtual space. 
     SUMMARY OF THE INVENTION 
     It is an object of the present invention to solve all the aforementioned problems. 
     It is another object of the present invention to allow a human in a present space and an avatar in the present space corresponding to another human in a remote place to naturally communicate with each other adaptively. 
     It is still another object of the present invention to naturally output an avatar&#39;s motion corresponding to a human&#39;s motion by reflecting characteristics of objects in the present space and the remote place. 
     In accordance with one aspect of the present invention, there is provided a telepresence system by using avatars, including: a corresponding location searching part for determining a location in a first space as a present space where an avatar Y′ corresponding to a human Y in a second space as a remote place will be placed, if a change of a location of the human Y in the second space is detected from an initial state in which an initial location of a human X in the first space, that of the human Y in the second space, that of the avatar Y′ in the first space corresponding to the human Y, and that of an avatar X′ in the second space corresponding to the human X are determined, by referring to (i) information on the first space and the second space and (ii) information on locations of the humans X and Y, and the avatar X′ in the first and the second spaces; and an avatar motion creating part for creating a motion of the avatar Y′ by referring to information on the determined location where the avatar Y′ will be placed. 
     In accordance with another aspect of the present invention, there is provided a method for providing telepresence by employing avatars, including steps of: (a) a corresponding location searching part determining a location in a first space as a present space where an avatar Y′ corresponding to a human Y in a second space as a remote place will be placed, if a change of a location of the human Y in the second space is detected from an initial state in which an initial location of a human X in the first space, that of the human Y in the second space, that of the avatar Y′ in the first space corresponding to the human Y, and that of an avatar X′ in the second space corresponding to the human X are determined, by referring to (i) information on the first space and the second space and (ii) information on locations of the humans X and Y, and the avatar X′ in the first and the second spaces; and (b) an avatar motion creating part creating a motion of the avatar Y′ by referring to information on the determined location where the avatar Y′ will be placed. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The above and other objects and features of the present invention will become apparent from the following description of preferred embodiments given in conjunction with the accompanying drawings, in which: 
         FIG. 1  is a concept diagram of a telepresence system in accordance with one example embodiment of the present invention. 
         FIG. 2  is a whole configuration of the telepresence system in accordance with one example embodiment of the present invention. 
         FIG. 3  represents a state of dividing spaces into patches to search corresponding locations under the telepresence system in accordance with the present invention. 
         FIG. 4  illustrates a state of information assigned to individual patches. 
     
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
     In the following detailed description, reference is made to the accompanying drawings that show, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It is to be understood that the various embodiments of the present invention, although different, are not necessarily mutually exclusive. For example, a particular feature, structure, or characteristic described herein in connection with one embodiment may be implemented within other embodiments without departing from the spirit and scope of the present invention. In addition, it is to be understood that the position or arrangement of individual elements within each disclosed embodiment may be modified without departing from the spirit and scope of the present invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is defined only by the appended claims, appropriately interpreted, along with the full range of equivalents to which the claims are entitled. In the drawings, like numerals refer to the same or similar functionality throughout the several views. 
     To allow those skilled in the art to the present invention to carry out easily, the example embodiments of the present invention by referring to attached diagrams will be explained in detail as follows: 
       FIG. 1  is a concept diagram under a telepresence system by using avatars in accordance with one example embodiment of the present invention. 
     By referring to the telepresence system of  FIG. 1 , an initial state where there are a human X in a first space as a present space, a human Y in a second space as a remote place, an avatar Y′ (corresponding to the human Y) in the first space, and an avatar X′ (corresponding to the human X) in the second space is represented. A method for determining the initial state will be explained later. 
     Under such a situation, if a change of a location of the human Y in the second space is detected, a corresponding location searching part  240  to be explained below may determine a location where the avatar Y′ will be placed in the first space by referring to (i) information on the first space and the second space and (ii) information on locations of the humans X and Y and the avatar X′ in the first and the second spaces. Herein, the corresponding location searching part  240  may also determine the location where the avatar Y′ will be placed in the first space by additionally referring to (iii) information on gazes of the humans X and Y in the first and the second spaces. Besides, an avatar motion creating part  250  to be explained later may create a motion of the avatar Y′ by referring to information on the location where the avatar Y′ will be placed. 
     Hereinabove, only the avatar Y′ was mentioned in detail, but it is natural that this could be applied equally even to the avatar X′. Further, the first space was mentioned as a present space and the second space was mentioned as a remote space but the first space could be the remote place and the second space could be the present space. However, for convenience, only one space will be explained. 
     For reference, the avatars X′ and Y′ may be virtual avatars visualized through head mounted displays (HMDs) worn by the humans X and Y and other augmented reality (AR) displays or physical avatars such as robots. At the time, appearances of the avatars could resemble humans at the remote places or be expressed as other figures such as animals or characters. 
     In addition, the present invention assumes a situation under which not only the structures of the present space and the remote place but also shapes, quantity, types, and layouts of objects such as furniture existing in the respective spaces could be different. 
     For example, if the human Y who stood in front of a door of the remote place approaches, and sits on, a chair at a different location in the remote place, the human Y at the remote place is allowed to be adapted to an environment of the present space by making the avatar Y′ used to represent the human Y take an action of standing in front of a door of the present space and then approaching a chair at a proper location in the present space. This will be explained later in more detail. 
     Furthermore, as explained above, as it was assumed that the structures of the present space and the remote place and the shapes, the quantities, the types, and the layouts of the objects in the present space and the remote place could be different, there could be a variety of routes of reaching the destination for the avatar Y′ in the present space according to the change of the motion of the human Y in the remote place. 
     For one example, even though the human Y at the remote place walked in a straight line to move to the chair from the front of the door, the avatar Y′ at the present space which has a structurally different environment from the remote place may take an action of turning around or getting over an obstacle to go to the chair from the front of the door. This will be also explained later in detail. 
     A whole configuration of the telepresence system by using avatars in the present invention based on the aforementioned concepts is illustrated in  FIG. 2 . 
     As shown in  FIG. 2 , a telepresence system  200  by using avatars in the present invention includes a space recognition part  210 , a human motion recognition part  220 , and a human gaze recognition part  230  for acquiring a variety of pieces of information to recognize actual conditions of a human and an environment, a corresponding location searching part  240  for searching a corresponding location, i.e., an optimal location where an avatar will be placed in the present space, an avatar motion creating part  250  for receiving a value from the corresponding location searching part  240  to map a motion of the human Y in the remote place to the avatar Y′ in the present space, a user interface providing part  260  for allowing a user to determine whether an estimated destination is appropriate or not, a communication part  270 , and a control part  280 . For reference, the telepresence system  200  in accordance with the present invention necessarily include the corresponding location searching part  240 , the avatar motion creating part  250 , the communication part  270 , and the control part  280 . Meanwhile, the space recognition part  210 , the human motion recognition part  220 , the human gaze recognition part  230 , and the user interface providing part  260 , if necessary, may be included in the telepresence system  200  in accordance with the present invention or separately in another system. 
     The space recognition part  210  measures data by means of various types of sensors, including an image sensor, a depth sensor, a laser sensor, etc. and acquires information on types of objects such as furniture (e.g., a chair, a table, a bed, etc.) and space information on locations and orientations of the objects (i.e., directions on which the objects are placed, etc.). 
     At the time, the space information may include information on types, locations, and orientations of the objects granted to the patches, which will be explained later. 
     To acquire information on a location of a human, the human motion recognition part  220 , moreover, could measure data with respect to a human motion by means of a wearable sensor such as an image sensor, a depth sensor, or an inertial measurement unit (IMU) and acquire information on a location and a motion of the human as a function of time through the measured sensor data. 
     Besides, the human gaze recognition part  230  acquires information on an object gazed by a human and a gaze point through a variety of sensor data with which gaze measurements are available. 
     The corresponding location searching part  240  calculates an appropriate location where the avatar Y′ in the present space corresponding to that of the human Y in the remote place will be placed or a changing location of the avatar Y′ in the present space, if the human Y in the remote place is changed his or her location within the remote place, based on the acquired information on the space, the location, the motion, and the gaze. 
     In other words, it calculates an optimal corresponding location in the present space where the avatar Y′ will be placed according to the movement of the human Y by receiving 1) information on the remote place and the present space, 2) information on locations of humans in the remote and the present spaces and/or 3) gaze information and then estimating a destination where the human in the remote place intends to approach. 
     At the time, the corresponding location searching part  240  may divide spaces and surfaces of objects into virtual patches, in which case the estimation of the destination and the calculation of the optimal corresponding location may be performed in a unit of patch. 
     The patches are created by dividing the spaces in a unit of plane. Herein, a curved surface may be divided into multiple planes. By determining a minimum value (e.g., 0.1 1 m 2 ) and a maximum value (e.g., 1 m 2 ) of an area of each patch to allow the patches to express a location of a human properly, a plane not more than the minimum value may be controlled not be included in the patch and that not less than the maximum value may be controlled to be divided into multiple patches. 
     The corresponding location searching part  240  gives and stores an ID of an object and geometrical information (including sizes, heights, and directions of normal vectors) with respect to each patch according to space information. Besides, each patch has information on a location and a principle direction in a space according to the space information. This will be stated by referring to  FIG. 3 . 
     By referring to  FIG. 3 , if s A, i  is a patch i corresponding to a space A, s A, p  corresponds with a patch P which corresponds to the space A and s A, q  corresponds with a patch Q which corresponds to the space A. 
     On the assumption that a location of s A, i  is p A, i  and a principal direction thereof is d A, i , if the corresponding patch has a principal direction such as a chair, the principal direction d A, i  is designated as a direction taken principally by a human and if the corresponding patch has no principal direction such as a floor, the principal direction d A, i  is designated as “undefined.” 
     As shown in  FIG. 3 , a principal direction d A, q  of a patch s A, q  corresponding to a chair is represented as an arrow head and a principal direction d A, p  of a patch s A, p  corresponding to a floor is undefined, which means that there is no corresponding arrow head. 
     Meanwhile, the corresponding location searching part  240  calculates an optimal corresponding location, i.e., a patch and a direction of the patch where an avatar should be placed at an initial state when the avatar is initially operated as shown below. This will be explained in more detail by referring to  FIG. 4 . 
     By referring to  FIG. 4 , on the assumption that a location and an orientation of the human X in the present space A are p A, i  and θ A, i  and those of the human Y in the remote place are p B, j  and θ B, j , an optimal patch s A, j′  and an optimal orientation θ A, j′  of the avatar Y′ (corresponding to the human Y in the remote place) to be placed in the present space are determined such that a degree of intrinsic similarity and a degree of spatial proximity reach maximum values. 
     The degree of intrinsic similarity represents a degree of similarity with respect to (i) information on types of objects (e.g., a chair, a table) in the patches and (ii) geometrical characteristics (e.g., normal vectors, patch sizes, heights, and aspect ratios) of the patches. 
     The degree of spatial proximity may be determined as a function which increases when p A, i  and p A, j′  are closer to a certain distance α (determined as a proper distance that people can talk with each other, e.g., 1 m) and θ A, j′i  (i.e., a difference between θ A, j′  and θ A, i ) is closer to 180 degrees to make a human and an avatar face each other. For example, information on the degree of spatial proximity may be determined as a function shown below.
 
( p   A,j′   ,p   A,i ,θ A,j′i )=exp(−λ 1 (∥ p   A,j′   p   A,i ∥−α)−λ 2 |θ A,j′i −180|)
 
     where λ i  (≧0) is a weighting factor which controls the influence of each term. 
     The initial optimal corresponding location may be determined as a location with high degrees of intrinsic similarity and spatial proximity. For example, the location may be determined as a location where the following fitness function is maximized:
 
max{(1− w   2 )·intrinsic_similarity( s   A,j′   ,s   B,j )+ w   2 ·spatial_proximity( p   A,j′   ,p   A,i ,θ A,j′i )}
 
     where w 2  (≧0) is a weighting factor which controls the influence of the intrinsic similarity and the spatial proximity. 
     Similarly, a location p B, i′  and orientation θ B, i′  of the avatar X′ in the remote place could be obtained by the aforementioned method. 
     Besides, it could be also assumed that, if the present and the remote places are divided into multiple patches, the corresponding location searching part  240  allows the avatar Y′ in the present space to have a direction of 180 degrees (or near 180 degrees) from a direction of the human X and have a location within a predetermined distance from the human X to thereby be placed in a patch with a high degree of appropriateness for a human to be located in the present space, and allows the avatar X′ in the remote place to have a direction of 180 degrees (or near 180 degrees) from the a direction of the human Y and have a location within a preset distance from the human Y to thereby be placed in a patch with a high degree of appropriateness for a human to be located in the remote place. 
     By the afore-stated method, the initial states of the locations of the human X and the avatar Y′ in the present space and the human Y and the avatar X′ in the remote place would be determined. As shortly mentioned above, the states in  FIG. 1  are examples of the initial states of the human X and the avatar Y′ in the present space and the human Y and the avatar X′ in the remote place. 
     Meanwhile, it is assumed that the human Y in the remote place changes his or her location from the initial state of the location of the human Y. In the case, the corresponding location searching part  240  may calculate an appropriate location where the avatar Y′ in the present space will be placed when it detects the changed location of the human Y in the remote place. 
     First of all, to calculate the appropriate location where the avatar Y′ in the present space will be placed, a degree of similarity between patches of the present space and those of the remote place should be calculated. 
     In other words, the degree of similarity between patches of the present space and those of the remote place is obtained by calculating a degree of intrinsic similarity and a degree of spatial similarity. 
     The degree of spatial similarity is acquired by referring to (i) information on a relative location and a relative direction between the human X and the avatar Y′ in the first space and (ii) information on a relative location and a relative direction between the human Y and the avatar X′ in the second space. 
     First of all, on the assumption that a human and an avatar are placed in two patches s A, i  and s A, j  in the space A in  FIG. 4  with specified orientations θ A, i  and θ A, j′  respectively (arrow heads in  FIG. 4 ), θ A, i  may be equal to or different from d A, i , or may be designated as undefined as the case may be. 
     At the time, a coordinate system may be set based on p A, i  and θ A, i  and a relative location of s A, j′  in reference to the set coordinate system is defined as p A, j′i  and a difference between the orientations θ A, i  and θ A, j′  in reference to the set coordinate system is defined as θ A, j′i  (=θ A, j′ −θ A, i ). If either of θ A, i  and θ A, j′  is undefined, θ A, j′i  may be determined as undefined. s B, ji′  and θ B, ji′  may be determined in the space B in the similar manner. 
     Accordingly, a degree of spatial similarity between s A, j′  and s B, j  with respect to reference patches s A, i  and s B, i′  is determined as a function which increases as the differences between p A, j′i  and p B, ji′  and between θ A, j′i  and θ B, ji′  become smaller. For example, the degree of spatial similarity may be calculated as follows:
 
spatial_similarity=exp(−λ 1   ∥p   A,j′i   −p   B,ji′ ∥−λ 2 |θ A,j′i −θ B,ji′ |)
 
     where λ i  (≧0) is a weighting factor which controls the influence of each term. 
     At the time, if both θ A, j′i  and θ B, ji′  are undefined, θ A, j′i −θ B, ji′  may be determined as 0 and if either of them is undefined, a sufficiently large value may be given to |θ A, j′i −θ B, ji′ |. 
     Next, a method of the corresponding location searching part  240  calculating an appropriate location where the avatar Y′ in the present space will be placed when a change of a location of the human Y in the remote place is detected will be explained in detail. 
     If detecting a change of location of the human Y in the remote place, the corresponding location searching part  240  estimates a location which the human Y (moving) in the remote place intends to reach, i.e., a patch (s B, j ) as a destination. 
     To estimate the patch (s B, j ) as the destination, the corresponding location searching part  240  may refer to information on probabilities of the human Y reaching the respective patches and then determine a patch with the highest probability as the patch (s B, j ), i.e., the destination. The probabilities P tp  (s B, k ) (k=1, 2, . . . ) of the human Y reaching the respective patches as the destination, i.e., s B, k  (k=1, 2, . . . ), may be obtained as shown below. Herein, the probabilities P tp  (s B, k ) (k=1, 2, . . . ) of the human Y reaching the respective patches as the destination means probabilities of the respective patches being determined as the destination which the human Y intends to reach.
 
 P   tp ( s   B,k )=gaze_term*affordance_prior_term*humanal_prior_term
 
     where the gaze term is a function which increases as a cumulative value (an integral value) of probabilities of gazing at a patch s B, k  during a time section (e.g., one second) from the past to the present increases. 
     Furthermore, the affordance prior term is a function for evaluating degrees of appropriateness of a human to be located by referring to types of objects to which the patches correspond and geometrical characteristics of the patches (e.g., heights, normal vector directions). For instance, the types of objects such as a chair, or a floor where the human is easy to be located have relatively higher values than those of objects such as a table or a desk. Further, if a normal direction of an object is more distant from the opposite direction of gravity, a relatively lower value may be assigned. Additionally, the affordance prior term has a smaller value when a height of an object is higher than a reference height difficult for the human to be located (e.g., 1.5 meters from the floor) or when an empty space on a patch is lower than a certain reference height (e.g., 1 meter). 
     In addition, the personal prior term is a function which increases as a frequency of a patch s B, k  being the destination actually reached by the human Y increases. 
     Accordingly, s B, j  could be selected as a patch with the largest P tp  among the patches in the space B (k=1, 2, . . . ), and it could be explained under the following formula:
 
 s   B,j =arg max k    P   tp ( s   B,k )
 
     Meanwhile, the corresponding location searching part  240  may deliver information on s B, j  as a patch with the largest P tp  as shown above through the user interface providing part  260  of the human Y moving in the remote place. At the time, the user interface providing part  260  may include an AR display such as an HMD and the human Y may give feedback about whether the estimation result, i.e., the information on s B, j , is appropriate or not by using the user interface provided by the user interface providing part  260 . For example, if s B, j  is not a destination the human Y intends to move and reach, the human Y may give a negative feedback through the user interface providing part  260  in a meaning of wrong estimation. Then, the corresponding location searching part  240  may estimate another destination candidate (until the human Y gives a positive feedback) through an iterative process, i.e., through a course of repeatedly delivering information on a patch with a next highest probability to the user interface providing part  260  until a positive feedback is acquired from the human Y. 
     Next, on assumption that the estimated destination patch for the human Y moving in the remote place is designated as s B, j , the location and the orientation of the avatar X′ in the remote place are p B, i′  and θ B, i′ , and the location and the orientation of the human X in the present space are p A, i  and θ A, i , an optimal patch s A, j′  and an optimal orientation θ A, j′  of the avatar Y′ (i.e., the avatar for the human Y in the remote place) can be calculated to thereby acquire an optimal corresponding location for the avatar Y′. 
     θ B, j  is determed as d B, j  (a principal direction of s B, j ) and if d B, j  is undefined, even θ B, j  is designated as undefined. 
     s A, j′  and θ A, j′  are selected such that the degree of intrinsic similarity and the degree of spatial similarity as specified above become maximums. 
     For example, s A, j′  and θ A, j′  are selected such that the following fitness function becomes maximum:
 
max{(1− w   1 )·intrinsic_similarity( s   A,j′   ,s   B,j )+ w   1 ·spatial_similarity( p   A,j′i   ,p   B,ji′ ,θ A,j′i ,θ B,ji′ )}
 
     w 1  (≧0) as a scalar is a weighting factor which controls degrees of importance with respect to the intrinsic similarity and the spatial similarity. 
     For reference, by the aforementioned optimization course, if θ B, j  is undefined, a patch whose θ A, j′  is undefined has a higher probability to be selected as s A, j′  and if θ b, j  is defined, a patch whose θ A, j′  is defined has a higher probability to be selected as s A, j′ . 
     Meanwhile, the avatar motion creating part  250  may create a motion of the avatar Y′ by referring to information on the estimated destination where the avatar Y′ will arrive. To do this, the avatar motion creating part  250  may be operated all the time to thereby create the movement of the avatar in the present space according to the movement of the human in the remote place. 
     The avatar motion creating part  250  outputs information on a shape of an avatar which takes a pose after receiving and analyzing at least some of 1) an output value of the corresponding location searching part  240 , 2) information on a human motion, a gaze and a space measured by recognition sensors (i.e., a human motion recognition part, a human gaze recognition part, and a space recognition part) in the remote place, 3) information on a human motion, a gaze and a space measured by recognition sensors in the present space, and 4) information on a basic shape (including body size, appearance, etc.) of the avatar. 
     When the corresponding location searching part  240  determines the destination where the avatar will be placed, the avatar motion creating part  250  searches an optimal route to make the avatar avoid interpenetration (i.e., collision) and creates a moving motion of the avatar. 
     There may be a lot of appropriate algorithms for searching a route and creating a moving motion. 
     At the time, the motion of the avatar may be created by referring to a type of the motion (e.g., sitting, standing, lying, or gesturing) of the human in the remote place. Due to a difference in shape between the human and the avatar and a difference in environment between the remote place and the present space, if a pose (e.g., angles of respective joints) of the human in the remote place is applied to the avatar in the present space as it is, the interpenetration with the environment (e.g., furniture, etc.) may occur. Therefore, the orientation of the avatar is determined to avoid the interpenetration with the environment while the orientation of the human in the remote place is kept to the maximum. 
     For example, a revised pose of the avatar may be determined by solving an optimization problem as shown below.
 
min{(1− w   3 )·pose_difference+ w   3 ·penetration_penalty}
 
     where pose_difference means a value which becomes larger when the difference in poses between the human and the avatar becomes larger and penetration_penalty means a value which becomes larger when a portion of the avatar penetrating into the environment becomes more. Of course, explanation on the algorithm for calculating a degree of penetration between two objects is omitted because there are many such algorithms. 
     In accordance with the present invention, if a corresponding location searching part displays a new destination where an avatar will be placed, the present invention has an effect of the avatar searching an optimal route to avoid interpenetration with the environment and then creating its moving motion. 
     The present invention also has an effect of creating a motion of an avatar in the present space corresponding to a type of a motion of a human in a remote place. 
     The embodiments of the present invention as explained above can be implemented in a form of executable program command through a variety of computer means recordable to computer readable media. The computer readable media may include solely or in combination, program commands, data files, and data structures. The program commands recorded to the media may be components specially designed for the present invention or may be usable to a skilled human in a field of computer software. Computer readable record media include magnetic media such as hard disk, floppy disk, and magnetic tape, optical media such as CD-ROM and DVD, magneto-optical media such as floptical disk and hardware devices such as ROM, RAM, and flash memory specially designed to store and carry out programs. Program commands include not only a machine language code made by a complier but also a high level code that can be used by an interpreter etc., which is executed by a computer. The aforementioned hardware device can work as more than a software module to perform the action of the present invention and they can do the same in the opposite case. 
     As seen above, the present invention has been explained by specific matters such as detailed components, limited embodiments, and drawings. While the invention has been shown and described with respect to the preferred embodiments, it, however, will be understood by those skilled in the art that various changes and modification may be made without departing from the spirit and scope of the invention as defined in the following claims. 
     Accordingly, the thought of the present invention must not be confined to the explained embodiments, and the following patent claims as well as everything including variations equal or equivalent to the patent claims pertain to the category of the thought of the present invention.