Patent Publication Number: US-2012044335-A1

Title: Makeup simulation system, makeup simulation apparatus, makeup simulation method, and makeup simulation program

Description:
TECHNICAL FIELD 
     The present invention relates to a makeup simulation system, a makeup simulation apparatus, a makeup simulation method and a makeup simulation program and, more particularly, to a makeup simulation apparatus, a makeup simulation method and a makeup simulation program for applying a make up to a face of a user contained in a dynamic picture image. 
     BACKGROUND ART 
     Conventionally, there is known a technique to simulate a face after makeup on a computer without actually applying a makeup for the purpose of sales of a commercial product for makeup (for example, refer to Patent Document 1). However, because a simulation result is displayed as a static image in the Patent Document 1, it is unable to easily check a face after makeup when facial expression of a user is changed. Thus, a technique of simulating makeup in a dynamic image, which acquires a change in facial expression of a user has being developed (for example, refer to Patent Document 2). 
     However, the makeup simulation apparatus disclosed in Patent Document 2 computes a makeup area for applying a face makeup by specifying a change in facial expression of a user in pixel areas corresponding to a mouth and both eyes and tracing the pixel areas according to template matching (for example, refer to paragraph 
     Such tracing of a change in facial expression of a user by pixel areas corresponding to a mouth and both eyes gives a large load to a computer, and there is a problem in that it is difficult to make an exact response to a case such as a case of closing eyes. 
     Patent Document 1: Japanese Laid-Open Patent Application No. 2001-346627 
     Patent Document 2: Japanese Laid-Open Patent Application No. 2003-44837 
     DISCLOSURE OF THE INVENTION 
     Problems to be Solved by the Invention 
     The present invention was made in view of the above-mentioned point, and it is an issue of the present invention to provide a makeup simulation system, a makeup simulation apparatus, a makeup simulation method and a makeup simulation program, which can correctly apply a makeup to a face of a user contained in a dynamic image at a small processing load. 
     Means to Solve Problems 
     In order to solve the above-mentioned problems, the present invention is a makeup simulation system for applying a makeup to a dynamic image of a face of a user, comprising: picture-taking means for taking a picture of the face of the user and outputting the dynamic image; control, means for receiving the dynamic image output from said picture-taking means, and image-processing and outputting said dynamic image; and display means for displaying the dynamic image output from said control means, wherein said control means includes: face recognition processing means for recognizing the face of the user from said dynamic image in accordance with predetermined tracking points; and makeup processing means for applying a predetermined makeup to the face of the user contained in said dynamic image in accordance with said tracking points, and outputting to said display means, wherein lipstick processing means included in said makeup processing means applies a lipstick process to the face of the user contained in said dynamic image in accordance with positions of a peak and a trough of an upper lip, left and light ends of a mouth, a lower end of a lower lip, and a position at ⅓ of a width of the mouth from the left end in the lower lip and a position at ⅓ of the width of the mouth from the right end in the lower lip. 
     According to the present invention, the face of the user can be recognized from the dynamic image based on the tracking points at a small processing load, and a correct makeup can be applied to the face of the user contained in the dynamic image based on the tracking points by being provided with the face recognition processing means for recognizing the face of the user from the dynamic image based on a predetermined tracking points and the makeup processing means for applying a predetermined makeup to the face of the user contained in the dynamic image based on the tracking points and outputting it to the display means. 
     It should be noted that a method, an apparatus, a system, a computer program, a recording medium, a data structure, etc., to which the structural elements and expressions of the present invention or an arbitrary combination of the structural elements are applied, may be effective as an illustrative embodiment of the present invention. 
     Effect of the Invention 
     According to the present invention, a makeup simulation system, a makeup simulation apparatus, a makeup simulation method and a makeup simulation program, which can correctly make up a face of a user contained in a dynamic image at a small processing load, can be provided. 
    
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         FIG. 1  is an external view of a first embodiment of a makeup simulation apparatus according to the present invention. 
         FIG. 2  is a cross-sectional view of the first embodiment of the makeup simulation apparatus. 
         FIG. 3  is an external view of a second embodiment of the makeup simulation apparatus according to the present invention. 
         FIG. 4  is a cross-sectional view of the second embodiment of the makeup simulation apparatus. 
         FIG. 5  is an external view of a third embodiment of a makeup simulation apparatus according to the present invention. 
         FIG. 6  is a cross-sectional view of the third embodiment of the makeup simulation apparatus. 
         FIG. 7  is an external view of a fourth embodiment of a makeup simulation apparatus according to the present invention. 
         FIG. 8  is a cross-sectional view of the fourth embodiment of the makeup simulation apparatus. 
         FIG. 9  is an external view of a fifth embodiment of a makeup simulation apparatus according to the present invention. 
         FIG. 10  is a cross-sectional view of the fifth embodiment of the makeup simulation apparatus. 
         FIG. 11  is a hardware structure diagram of the fifth embodiment of the makeup simulation apparatus. 
         FIG. 12  is a flowchart illustrating an outline of a process which the makeup simulation apparatus performs. 
         FIG. 13  is an image view of an example of an operation screen displayed on a main screen displayed on a monitor and a control panel. 
         FIG. 14  is an image view expressing a process other than a makeup simulation which the makeup simulation apparatus performs. 
         FIG. 15  is a system structure diagram of a makeup simulation system according to the present invention. 
         FIG. 16  is a screen image view expressing a process which a simulator main application performs. 
         FIG. 17  is an image picture of a face of a user contained in an image before makeup. 
         FIG. 18  is an image view of an example illustrating tracking points. 
         FIG. 19  is an illustration of a structure of an example of a makeup processing parameter file. 
         FIG. 20  is an image view indicating tracking points referred to by a lipstick process. 
         FIG. 21  is a flowchart of an example indicating the lipstick process. 
         FIG. 22  is an image view of an example illustrating a profile extraction process. 
         FIG. 23  is an illustration comparing tracking points of lip eight points of and nose three points of a contour extracting image  600  with points of a contour extracting image  601  extracted by re-searching based on the tracking points of lip eight points and nose three points. 
         FIG. 24  is an image view of an example illustrating a process of acquiring default points from the tracking points of lip eight points and nose three points. 
         FIG. 25  is an image view illustrating a process of completing a contour from the points or the default points. 
         FIG. 26  is an image view illustrating a color painting map creating process. 
         FIG. 27  is an image view of an example illustrating the color painting process based on a color painting map. 
         FIG. 28  is an image view illustrating the tracking points referred to by a shadow process. 
         FIG. 29  is a flowchart of an example illustrating the shadow process. 
         FIG. 30  is an image view of an example illustrating a creating process of a basic contour. 
         FIG. 31  is an image view of an example illustrating a creating process of a color painting contour. 
         FIG. 32  is an image view of an example illustrating a color painting map applied with no gradation process and a color painting map applied with a gradation process. 
         FIG. 33  is an image view illustrating tracking points referred to by a cheek process. 
         FIG. 34  is a flowchart of an example illustrating the cheek process. 
         FIG. 35  is an image view of an example illustrating a color painting contour. 
         FIG. 36  is an image view illustrating tracking points referred to by an eyebrow process. 
         FIG. 37  is a flowchart of an example illustrating the eyebrow process. 
         FIG. 38  is an image view of an example illustrating an eyebrow contour extracting process. 
         FIG. 39  is an image view of an example illustrating a creating process of a designation curve corresponding to a variation designation. 
         FIG. 40  is an image view illustrating tracking points referred to by a foundation process. 
         FIG. 41  is a flowchart of an example of the foundation process. 
         FIG. 42  is an image view of an example illustrating a contour. 
         FIG. 43  is an image view of an example illustrating an original image before a gradation process and an original image after a gradation process. 
         FIG. 44  is a flowchart of an embodiment of a lipstick process (lip pen drawing process). 
         FIG. 45  is a flowchart of an embodiment of a lipstick process (lip pen drawing process). 
         FIG. 46  is an image view of an example of a face image of a rectangular area surrounding a mouth. 
         FIG. 47  is an illustration illustrating drawing images. 
         FIG. 48  is an illustration illustrating an offset curve. 
         FIG. 49  is an image view of an example illustrating positions of a plurality of previously set points to grasp morphological features of lips. 
         FIG. 50  is a flowchart of an embodiment of an eyeline searching process. 
         FIG. 51  is a flowchart of an embodiment of an eyeline drawing process. 
         FIG. 52  is an illustration illustrating an eyeline drawing pattern. 
         FIG. 53  is an illustration illustrating an eyeline drawing pattern. 
         FIG. 54  is an illustration illustrating an eyeline drawing pattern. 
         FIG. 55  is an illustration illustrating an eyeline drawing pattern. 
         FIG. 56  is a flowchart of an embodiment of an eye shadow painting process. 
         FIG. 57  is an illustration for explaining setting of an eye shadow painting area. 
         FIG. 58  is an illustration illustrating an upper limit of the eye shadow painting area. 
         FIG. 59  is an illustration for explaining eye shadow painting. 
         FIG. 60  is an illustration illustrating a relationship between a distance from a center and a concentration in an air-brush process. 
         FIG. 61  is a front view illustrating a shape of an eye of a standard balance. 
         FIG. 62  is a flowchart of an embodiment of a rouge applying process (cheek process). 
         FIG. 63  is an illustration for explaining a rouge applying process in a case where a makeup pattern is sweet. 
         FIG. 64  is an illustration for explaining a rouge applying painting process in a case where a makeup pattern is cool. 
         FIG. 65  is an illustration for explaining a rouge applying process in a case where a makeup pattern is cute. 
         FIG. 66  is an illustration for explaining a rouge applying process in a case where a makeup pattern is fresh. 
         FIG. 67  is an illustration illustrating a relationship between a distance from a center and a concentration in an air-brush process. 
         FIG. 68  is an image view for explaining an example of a cheek makeup according to a morphological feature of a cheek of a person to be applied with makeup. 
         FIG. 69  is an explanatory illustration for explaining a form of an ideal eyebrow. 
         FIG. 70  is an explanatory illustration for explaining states of position of eyebrows. 
         FIG. 71  is a flowchart illustrating an example of a skin color evaluating process procedure as a pretreatment. 
         FIG. 72  is an illustration illustrating an example of feature points and divided areas. 
         FIG. 73  is an illustration illustrating an example of a positional relationship between the 109 feature points within a face corresponding to the above-mentioned  FIG. 72 . 
         FIG. 74  is an illustration illustrating an example of a combination of feature points constituting areas corresponding to the above-mentioned  FIG. 72 . 
         FIG. 75  is an illustration illustrating an example of features of color of each group and area numbers constituting each group. 
         FIG. 76  is an illustration illustrating an example of a result of analysis of an image (or stereoscopic effect) of an average face. 
     
    
    
     EXPLANATION OF REFERENCE NUMBERS 
     
         
         
           
               1  makeup simulation apparatus 
               2  camera 
               3  half mirror 
               4  operation panel 
               5  printer 
               6  lighting 
               7  monitor 
               8  control part 
               9  transparent plate 
               10  operation processing unit 
               11  memory device 
               12  drive unit 
               13  auxiliary memory device 
               14  recording medium 
               15  touch-panel monitor 
               16  exhibition case 
               17  IC-tag reader writer 
               25  static image system 
               27  shared memory 
               28  simulator main application 
               29  interface application 
               33  face recognition processing part 
               34  tracking point 
               35  makeup processing part 
               36  foundation processing part 
               37  eyebrow processing part 
               38  shadow processing part 
               39  lipstick processing part 
               40  cheek processing part 
               41  product information 
           
         
       
    
     BEST MODE FOR CARRYING OUT THE INVENTION 
     Next, a description will be give, with reference to the drawings, of a bet mode for carrying out the invention based on the embodiments mentioned below.  FIG. 1  is an external view of a first embodiment of a makeup simulation apparatus according to the present invention. The makeup simulation apparatus  1  is configured to include a camera  2 , an operation panel  4 , a printer  5 , a lighting  6 , and a monitor  7 . 
     The camera  2  take a picture of a user standing in front of the makeup simulation apparatus  1 , and outputs a dynamic image. The operation panel  4  displays an operation image, and receives an operation from the user and outputs operation information. The printer  5  prints an image (for example, an image picture after makeup, etc.) and information (for example, product information for make up like an image picture, etc.) which are displayed on a monitor. The lighting  6  performs a lighting adjustment after, for example, a makeup simulation is started. 
       FIG. 2  is a cross-sectional view of the first embodiment of the makeup simulation apparatus. The makeup simulation apparatus  1  is configured to include the camera  2 , the operation panel  4 , the printer  5 , the monitor  7 , and a control part  8 . The camera  2  outputs the picture-taken dynamic image to the control section  8 . The operation panel  4  displays an operation image output from the control part  8 , and receives an operation from a user and outputs operation information to the control part  8 . The monitor  7  displays the dynamic image (main image) output from the control part  8 . The control part  8  receives the dynamic image output from the camera  2 , and applies a makeup to the face of the user contained in the dynamic image and output it to the monitor  7  by image-processing the dynamic image as mentioned later. 
     The makeup simulation apparatus of  FIG. 1  and  FIG. 2  includes a mirror, which is an indispensable item for a makeup, as a conceptual structure, and has a user interactive function. That is, the makeup simulation apparatus  1  has a feature to make a user to feel as if the user is doing a makeup naturally while viewing a mirror. 
     The makeup simulation apparatus  1  applies, by the control part  8 , an image process to the dynamic image or the like output from the camera  2 , and displays it on the monitor  7 , which is a digital mirror, after applying a makeup to the face of the user contained in the dynamic image. The makeup simulation apparatus  1  is capable of displaying on the monitor  7  various sets of product information and cosmetic information and an image picture in which a make up has been applied to a face of a user. 
     Moreover,  FIG. 3  is an external view of a second embodiment of the makeup simulation apparatus according to the present invention. It is noted that parts the same as the parts of  FIG. 1  are given the same reference numerals. 
     The makeup simulation apparatus  1  of  FIG. 3  is configured to include the operation panel  4 , the printer  5 , the lighting  6  and a transparent plate  9 . The transparent plate  9  transmits a light from outside the makeup simulation apparatus, and also transmits a light from inside the makeup simulation apparatus  1 . 
       FIG. 4  is a cross-sectional view of the second embodiment of the makeup simulation apparatus. It is noted that parts that are the same as the parts in  FIG. 2  are given the same reference numerals. The makeup simulation apparatus  1  of  FIG. 4  is configured to include the camera  2 , a half mirror  3 , the operation panel  4 , the printer  5 , the monitor  7 , the control part  8 , and the transparent board  9 . 
     The half mirror (translucence mirror)  3  reflects a light incident thereon and also transmits a part of the light therethrough. The camera  2  is arranged at a position where the camera  2  can take a picture of a user standing in front of the makeup simulation apparatus  1  through the half mirror  3  and the transparent plate  9 . The camera  2  is arranged at a level of eyes of the user. The camera  2  takes a picture of the user standing in front of the makeup simulation apparatus  1  through the half mirror  3  and the transparent plate  9 , and outputs a dynamic image. 
     The monitor  7  is arranged at a position where the user standing in front of the makeup simulation apparatus  1  can see through the half mirror and the transparent plate  9 . The light output from the monitor  7  is reflected by the half mirror, and is output to outside the makeup simulation apparatus  1  through the transparent plate  9 . Accordingly, the user can see a dynamic image displayed on the monitor  7  from outside the makeup simulation apparatus  1 . 
     The makeup simulation apparatus  1  of  FIG. 3  and  FIG. 4  includes a mirror, which is indispensable for a makeup, as a conceptual structure, and has a user interactive function. That is, the makeup simulation apparatus  1  has a feature to make a user to feel as if the user is doing a makeup naturally while viewing a mirror. 
     Because the camera  2  is arranged at a level of eyes, the makeup simulation apparatus  1  is capable of taking a picture of the face of the user standing in front of the makeup simulation apparatus naturally rather than it is arranged at the position of the camera  2  as in the first embodiment. 
     The makeup simulation apparatus  1  applies, by the control part  8 , an image process to a dynamic image or the like output from the camera  2  to apply a makeup to the face of the user contained in the dynamic image, and displays it on the monitor  7 , which is a digital mirror. The makeup simulation apparatus  1  can display on the monitor  7  various sets of product information and cosmetic information and an image view of the face of the user to which a makeup has been applied. 
     Moreover,  FIG. 5  is an external view of a third embodiment of the makeup simulation apparatus according to the present invention. It is noted that parts that are the same as the parts in the previous figures are given the same reference numerals. The makeup simulation, apparatus  1  of  FIG. 5  is configured to include the camera  2 , the one-way mirror  3 , the operation panel  4 , the printer  5 , and the lighting  6 . 
     The half mirror  3  transmits a part (for example, 50%) of a light from the brighter side, and reflects a reminder (for example, 50%) of the light from the brighter side to the front side. Because there is no light on the darker side, the half mirror does not transmit or reflect a light from the darker side. 
     After a makeup simulation is started, the lighting  6  adjusts a light so that a side of the monitor  7  of the half mirror  3  becomes bright. Therefore, before the makeup simulation is started, the half mirror  3  provided on the display side of the monitor  7  reflects a light from a user side (outside the makeup simulation apparatus  1 ) in order to function as a mirror. 
     After the makeup simulation is started, the half mirror  3  transmits a light from the monitor  7  side (inside the makeup simulation apparatus  1 ) in order to function as glass. Therefore, the user can see a dynamic image displayed on the monitor through the half mirror  3 . 
       FIG. 6  is a cross-sectional view of a third embodiment of the makeup simulation apparatus. The makeup simulation apparatus  1  of  FIG. 6  is configured to include the camera  2 , the half mirror  3 , the operation panel  4 , the printer  5 , the monitor  7 , and the control part  8 . It is noted that parts that are the same as the parts in the previous figures are given the same reference numerals. The dynamic image displayed on the monitor  7  transmits through the half mirror  3 , which functions as glass. The user can see the dynamic image displayed on the monitor  7  from outside the makeup simulation apparatus  1 . 
     The makeup simulation apparatus  1  of  FIG. 5  and  FIG. 6  includes a mirror, which is indispensable for a makeup, as a conceptual structure, and has a user interactive function. That is, the makeup simulation apparatus  1  has a feature to make a user to feel as if the user is applying a makeup naturally while viewing a mirror. 
     The makeup simulation apparatus  1  applies, by the control part  8 , an image process to a dynamic image or the like output from the camera  2  to apply a makeup to the face of the user contained in the dynamic image, and displays it on the monitor  7 , which is a digital mirror. The makeup simulation apparatus  1  can display on the monitor  7  various sets of product information and cosmetic information and an image view of the face of the user to which a makeup has been applied. 
     Moreover,  FIG. 7  is an external view of a fourth embodiment of the makeup simulation apparatus according to the present invention.  FIG. 8  is a cross-sectional view of the fourth embodiment of the makeup simulation apparatus. It is noted that parts that are the same as parts in the previous figures are given the same reference numerals. The makeup simulation apparatus  1  is configured to include the camera  2 , the printer  5 , the lighting  6 , and a touch-panel monitor  15 . 
     The makeup simulation apparatus  1  of  FIG. 7  and  FIG. 8  differs from the makeup simulation apparatus  1  of  FIG. 1  and  FIG. 2  in that it is configured to include the touch-panel monitor  15  instead of the operation panel  4  and the monitor  7 . The touch-panel monitor  15  functions as the operation panel  4  and the monitor  7 . 
     The touch-panel monitor  15  displays an operation image, and receives an operation from a user and outputs operation information. The touch-panel monitor  15  displays the operation image output from the control part  8 , and receives an operation from a user and outputs the operation information to the control part  8 . The touch-panel monitor  15  displays a dynamic image (main image) output from the control part  8 . The control part receives the dynamic image output from the camera  2 , and applied a makeup to the face of the user contained in the dynamic image by image-processing the dynamic image as mentioned later, and outputs it to the touch-panel monitor  15 . 
     The makeup simulation apparatus  1  of  FIG. 7  and  FIG. 8  includes a mirror, which is indispensable for a makeup, as a conceptual structure, and has a user interactive function. That is, the makeup simulation apparatus  1  has a feature to make a user to feel as if the user is applying a makeup naturally while viewing a mirror. 
     The makeup simulation apparatus  1  applies, by the control part  8 , an image process to a dynamic image or the like output from the camera  2  to apply a makeup to the face of the user contained in the dynamic image, and displays it on the touch-panel monitor  15 , which is a digital mirror. The makeup simulation apparatus  1  can display on the touch-panel monitor  15  various sets of product information and cosmetic information and an image view of the face of the user to which a makeup has been applied. 
     The makeup simulation apparatus  1  of the fourth embodiment may be provided with an exhibition case for exhibiting products for testing, which a user uses and tests, as illustrated in  FIG. 9  and  FIG. 10  in order to use it as a case for self-sales.  FIG. 9  is an external view of a fifth embodiment of the makeup simulation apparatus according to the present invention.  FIG. 10  is a cross-sectional view of the fifth embodiment of the makeup simulation apparatus. It is noted that parts that are the same as previous figures are given the same reference numeral. 
     The makeup simulation apparatus  1  is configured to include the camera  2 , the printer  5 , the lighting  6 , the touch-panel monitor  15 , an exhibition case  16 , and an IC-tag leader writer  17 . The makeup simulation apparatus  1  of  FIG. 9  and  FIG. 10  differs from the makeup simulation apparatus  1  of  FIG. 7  and  FIG. 8  in that it includes the exhibition case  16  and the IC-tag leader writer  17 . 
     The exhibition case  16  is for exhibiting a plurality of products for testing. It should be noted that an IC tag (RFID) is attached to each product for testing. Identification information which can identify each product for testing is stored in the IC tag attached to the product for testing. When a user takes one of the products for testing out of the exhibition case  16  and move it to close to the IC-tag leader writer  17 , the IC-tag leader writer  17  reads the identification information on the product for testing. 
     The IC tag leader writer  17  transmits the identification information of the product for testing read from the IC tag to the control part  8 . The control part  8  receives a dynamic image output from the camera  2 , and outputs to the touch-panel monitor  15  an image picture in which a makeup is applied to the face of the user contained in the dynamic image by using the product for testing corresponding to the identification information read from the IC tag. 
     It should be noted that a correspondence table for relating the products for testing and the identification information may be provided to the makeup simulation apparatus  1 , or provided to other apparatuses which are capable of acquiring the correspondence table through a network. 
     Moreover, although an example of using the IC tag to identify each product for testing in the makeup simulation apparatus  1  of  FIG. 9  and  FIG. 10  was indicated, a bar code, a two-dimensional code, a label, etc., may be used. Further, the makeup simulation apparatus  1  of  FIG. 9  and  FIG. 10  may be provided with a device, which can identify each product for testing taken out, in the exhibition case  16  so that a notification is sent from the exhibition case  16  to the control part  8 . 
     The makeup simulation apparatus  1  of  FIG. 9  and  FIG. 10  includes a mirror, which is indispensable for a makeup, as a conceptual structure, and has a user interactive function. That is, the makeup simulation apparatus  1  has a feature to make a user to feel as if the user is applying a makeup naturally while viewing a mirror. 
     The makeup simulation apparatus  1  applies, by the control part  8 , an image process to a dynamic image or the like output from the camera  2  to display an image picture on the touch-panel  15 , which is a digital mirror, in which image picture a make up is applied to the face of the user contained in the dynamic image by using the product for testing, which the user selects from the products in the exhibition case  16 . The makeup simulation apparatus  1  can display on the touch-panel monitor  15  various sets of product information and cosmetic information and an image view of the face of the user to which a makeup has been applied. The makeup simulation apparatus  1  can acquire data of user preference or the ling by taking a log of the product for testing selected from the products in the exhibition case  16 . 
     It should be noted that if the makeup simulation apparatus  1  is provided with a shelf for exhibiting not only the products for testing but also commercial products, it can be effectively used as a case for self-sales by exhibiting the commercial products displayed on the touch-panel monitor  15 . 
     Hereafter, a description will be given below of the makeup simulation apparatus  1  of the first embodiment as an example.  FIG. 11  is a hardware structure diagram of an embodiment of the makeup simulation apparatus. It is noted that parts the same as parts in  FIG. 1  and  FIG. 2  are given the same reference numerals. 
     The makeup simulation apparatus  1  of  FIG. 11  is constituted by the camera  2 , the operation panel  4 , the printer  5 , the monitor  7 , an operation processing device  10 , a memory device  11 , a drive unit  12 , and an auxiliary memory storage  13 , which are mutually connected by a bus B. The operation processing device  10 , the memory device  11 , the drive unit  12 , and the auxiliary memory storage  13  of  FIG. 11  constitute the control part  8  of  FIG. 2 . 
     A makeup simulation program of the present invention is at least a part of various programs for controlling the makeup simulation apparatus  1 . The makeup simulation program is provided by distribution of, for example, a recording medium  14 . 
     It should be noted that various types of recording media can be used as the recording medium  14  on which the makeup simulation program is recorded, such as a recording medium for recording information optically, electrically or magnetically such as a CD-ROM, a flexible disk, a magneto-optical disc, etc., and a semiconductor memory for recording information electrically such as a ROM, flash memory, etc. 
     Moreover, when the recording medium  14  on which the makeup simulation program is recorded is set to the drive unit  12 , the makeup simulation program is installed into the auxiliary memory storage  13  through the drive unit  12  from the recording medium  14 . The auxiliary memory storage  13  stores a required file, data, etc., while storing the installed makeup simulation program. The memory device  11  reads the makeup simulation program from the auxiliary memory storage  13  and stores it therein at the time of start up. Then, the operation processing device  10  realizes various processes mentioned later according to the makeup simulation program stored in the memory device  11 . 
       FIG. 12  is a flowchart of an embodiment illustrating an outline of a process, which the makeup simulation apparatus performs.  FIG. 13  is an image view of a main screen displayed on a monitor and an operation screen displayed on the operation panel. The screen images  100 - 111  of  FIG. 13  are for a main screen displayed on the monitor  7 . The screen images  200 - 210  are for an operation screen displayed on the operation panel  4 . 
     The control part  8  continuously receives the dynamic image taken by the camera  2 . At this time, the control part  8  displays the screen image  100  on the monitor  7 , and displays the screen image  200  on the operation panel  4 . The screen images  100  and  200  represent an example of displaying a screen saver. 
     Proceeding to step S 1 , the control part  8  continuously determines whether or not a face of a user is contained in the dynamic image which has received. The control part  8  repeats the process of step S 1  until it is recognized that the face of the user is contained in the dynamic image (NO in step S 1 ). 
     If it is recognized that the face of the user is contained in the dynamic image (YES in S 1 ), the control part  8  proceeds to step S 2  to activate software including the makeup simulation program for performing a makeup simulation. At this time, the control part  8  displays the screen image  101  on the monitor  7 , and displays the screen image  201  on the operation panel  4 . 
     The screen image  101  represents an example of displaying the dynamic image in which the face of the user is taken by the camera  2 . The screen image  201  represents an example of displaying a welcome comment moving horizontally. 
     Proceeding to step S 3 , the control part  8  starts a makeup simulation according to the software activated in step S 2 . At this time, the control part  8  displays the screen image  102  on the monitor  7 , and displays the screen image  202  on the operation panel  4 . The screen image  102  represents an example of displaying the dynamic image in which the face of the user is taken by the camera  2  with staging like a magic mirror. The screen image  202  represents an example of displaying a welcome comment moving horizontally, similar to the screen image  201 . 
     Proceeding to step S 4 , the control part  8  performs a makeup simulation as mentioned later. At this time, the control part  8  sequentially displays the screen images  103 - 106  on the monitor  7 , and displays the screen images  203 - 206  on the operation panel  4 . 
     The screen images  103 - 106  represent an example of displaying the face of the user to which four makeup patterns (images) are applied according to the makeup simulation. The screen images  203 - 206  represent an example of displaying contents (for example, designation or the like) of the makeup patterns of the image screens  103 - 106  displayed on the monitor  7  at that time. The control part  8  sequentially displays the screen images  103 - 106  on the monitor  7  and displays the screen images  203 - 206  on the operation panel until a predetermined time has passed or the user touches the operation panel  4 . 
     If a predetermined time has passed or the user touches the operation panel  4 , the control part  8  proceeds to step S 5  to display the screen image  107  on the monitor  7  and displays the screen image  207  on the operation panel  4 . The screen image  107  represents an example of displaying the dynamic image in which the face of the user is taken by the camera  2 . The screen image  207  represents an example of displaying an image selection screen through which one image is selectable from four makeup patterns (images). The user can select one image through the image selection screen by operating the operation panel  4 . 
     Proceeding to step S 6 , the control part  8  repeats the process of step S 6  until one image is selected through the image selection screen (NO in S 6 ). If the user selects one image through the image selection screen, the operation panel  4  receives an operation of the user and outputs operation information to the control part  8 . 
     If it is determined that the user selects one image through the image selection screen (YES in S 6 ), the control part  8  proceeds to step S 7  to display the image screen of the selected image on the monitor  7  and the operation panel  4 . At this time, the control part  8  displays the screen image  108  on the monitor  7  and displays the screen image  208  on the operation panel  4 . 
     The screen image  108  represents an example of sequentially displaying an image picture of the face of the user after a different color pattern is applied and product information for making up like the image pictures. The screen image  208  represents an example of displaying contents of the elected one of images selected through the image selection screen and product information for making up like the image picture of the screen image  108  displayed on the monitor at that time. 
     It should be noted that the user can also instruct a print-out by operating the operation panel  4 . Upon receipt of the print-out instruction from the user, the control part  8  displays the screen image  109  on the monitor  7  and displays the screen image  209  on the operation panel  4 . The screen image  109  represents an example of displaying an image picture of printing out. The screen image  209  represents an example of displaying a comment being printed out. The control part  8  prints out the image picture displayed on the monitor by controlling the printer  5 . 
     Moreover, the user can also instruct a display and print-out of a comparison screen, which includes image pictures before and after the makeup by operating the operation panel  4 . Upon receipt of the instruction of displaying the comparison screen, the control part displays the screen image  110  on the monitor  7 . The screen image  110  represents an example of displaying the comparison image including the image pictures before and after the makeup. Upon receipt of the instruction of printing out from the user in the state where the comparison screen is displayed, the control part  8  prints out the comparison screen displayed on the monitor  7  by controlling the printer  5 . 
     After the makeup simulation by the user is completed, the control part  8  displays the screen images  111  and  210 , which are screen savers, on the monitor  7  and the operation panel  4 , and the process is ended. 
     It should be noted that although the four images are displayed according to the makeup simulation in the example of  FIG. 12  and  FIG. 13 , images other than the four images may be displayed. Additionally, although the makeup simulation apparatus  1  illustrates the example in which a makeup is applied to the face of the user contained in the dynamic image output from the camera  2 , a previously taken dynamic image may be stored as a dynamic image file in the auxiliary memory storage  13  or the like and a makeup may be applied to a face of a user contained in the dynamic image file. 
     The makeup simulation apparatus  1  can also use the dynamic image output from the camera  2  to a process other than the makeup simulation.  FIG. 14  is an image view illustrating a process other than the makeup simulation which the makeup simulation apparatus performs. 
     A screen image  300  represents an example of displaying a dynamic image of a face of a user taken by the camera  2 . The makeup simulation apparatus  1  recognizes the face of the user contained in the dynamic image, and extracts a static picture  301  of the face of the user. 
     The makeup simulation apparatus  1  performs a face diagnosis and a skin color diagnosis of the static picture  301  according to a feature analysis logic and a skin color analysis logic, and displays a screen image  302  representing a result thereof on the monitor  302 . It should be noted that the process of performing a face diagnosis and a skin color diagnosis of the static picture  301  according to the feature analysis logic and the skin color analysis logic is a known technique disclosed in, for example, Japanese Laid-Open Patent Application No. 2001-346627. 
     Moreover, the makeup simulation apparatus  1  can display a course selection screen  303  like the screen image  303  on the monitor  7  to let the user to make a selection of each course (for example, trend, basic, free). The makeup simulation apparatus  1  displays screen images  304 - 309  based on the course selected by the user on the monitor  7 , and performs a simulation and advice. 
     The screen images  304 - 306  represent and example of displaying a simulation screen of each course. The screen images  307 - 309  represent an example of displaying an advice screen of each course. 
     For example, the basic course is a course for simulating and advising an appropriate technique in accordance with results of the feature diagnosis and the skin color diagnosis. Additionally, the trend course is a course for simulating and advising a latest trend makeup. Additionally, the free makeup course is a course for simulating and advising about an item corresponding to an eye, a mouth, a cheek, an eyebrow, and an individual part. 
     Upon receipt of the print-out instruction from the user in a state where the simulation screen or the advice screen is displayed, the control part  8  can also print out the simulation screen or the advice screen, which is displayed on the monitor  7 , by controlling the printer  5 . 
     Next, a description will be give of details of the makeup simulation system for realizing the above-mentioned makeup simulation apparatuses  1 .  FIG. 15  is a system structure diagram of an embodiment of the makeup simulation system according to the present invention. 
     The makeup simulation system  20  of  FIG. 15  is configured to include an analog camera  21 , a USB capture device  22 , a dynamic image file  23 , a makeup camera  24 , a static image system  25 , a dynamic image file  26 , a shared memory  27 , and a simulator main application  28  and an interface application  29 . The analog camera  21  outputs a dynamic image of, for example, an NTSC system via the USB capture device  22 . The dynamic image output from the USB capture device  22  is input to the simulator main application  28  by using a DirectX  31  as an example of API (application programming interface). Additionally, the dynamic image file  23  is also input to the simulation main application by using the DirectX  31 . 
     The makeup camera  24  outputs the taken dynamic image via an IEEE1394 as an example of a serial interface. The dynamic image output from the makeup camera  24  is input to the simulator main application  28  by using an exclusive API  32 . The simulator main application  28  acquires an original image of a resolution for dynamic image and a static image of a resolution for static picture by using the exclusive API  132 . 
     The simulator main application  28  sets the dynamic image and the dynamic image file  23  input by using the DirectX  31  and the dynamic image of a resolution for dynamic image acquired from the dynamic image input by using the exclusive API  32  as original images, and applies a trimming and a reduction process to the original images. 
     The simulator main application  28  applies the trimming to the original images to obtain a pre-makeup image. Additionally, the simulator main application  28  applies the reduction process to the original images to obtain a face recognition processing image. A face recognition processing part  33  acquires tracking points  34  mentioned later for recognizing a face of a user from the face recognition processing image according to an FFT (Fast Fourier Transform). 
     The makeup processing part  35  applies a makeup, which includes a foundation, an eyebrow, a shadow, a lipstick, and a cheek, to the face of the user contained in the pre-makeup image in accordance with the tracking points to obtain a post-makeup image. The makeup processing part  35  is configured to include a foundation processing part  36 , an eyebrow processing part  37 , a shadow processing part  38 , a lipstick processing part  39  and a cheek processing part  40 . 
     It should be noted that the makeup processing part  35  can include product information  41  for making up like the post-makeup image in the post-makeup image. A dynamic image server  42  writes the pre-makeup image and the post-makeup image in the shared memory  27 , and can output the pre-makeup image and the post-makeup image as the dynamic image file  26 . 
     The interface application  29  is configured to include a dynamic image control object  52 , a dynamic image display object  53  and controller  54 , which use an ActiveX controller  50  and an ActiveX viewer  51 . It should be noted that the interface application  29  and the simulator main application  28  attempt to interact with each other. 
     The interface application  29  can display the pre-makeup image and the post-makeup image written in the shared memory  27  on the previously mentioned monitor  7  by using the ActiveX viewer  51 . 
     The simulator main application  28  applies a trimming and a reduction process to the static image of a resolution for static image acquired from the dynamic image input using the exclusive API  32 . The simulator main application  28  applies a trimming to the static image of a resolution for static image. Further, the simulator main application  28  obtains a face recognition processing image by applying a reduction process to the static image of a resolution for static image. A face recognition processing part  43  acquires tracking points as mentioned later for recognizing a face of a user from the face recognition processing image. 
     The simulator main application  28  extracts a detail portion from the face of the user contained in the trimmed dynamic image in accordance with the tracking points  44 , and outputs to the static image system  25  tracking points  45 , which is the tracking points  44  added with additional information, and the static image of a resolution for static image acquired from the dynamic image input by using the exclusive API  32 . 
     The static image system  25  performs the feature diagnosis and the skin color diagnosis of the static image  301  by using the tracking points  45  and in accordance with the above-mentioned feature analysis logic and the skin color analysis logic, and can display a screen image  302  representing the result thereof on the monitor  7 . Besides, the static image system  25  can display screen images  303 - 309  on the monitor  7 . 
       FIG. 16  is a screen image view illustrating a process which the simulator main application performs. A screen image  400  represents an example of displaying a pre-makeup image. A screen image  401  represents an example of a display in which tracking points  34  obtained from the face recognition processing image are superimposed on the pre-makeup image. Moreover, a screen image  402  represents an example of displaying a post-makeup image which is the face of the user contained in the pre-makeup image to which a makeup has been applied according to the tracking points  34 . 
     A description will be sequentially given below, with reference to the drawings, of details of a face recognition process and a makeup process from among processes performed by the simulator main application  28 . It should be noted that although a foundation process, an eyebrow process, a lipstick process and a cheek process are explained as an example of the makeup process in the present embodiment, other combinations may be used. 
     (Face Recognition Process) 
       FIG. 17  is an image view of a face of a user contained in the pre-makeup image.  FIG. 18  is an image view of an example of displaying tracking points. A face recognition processing part  33  of the simulator main application  33  acquires  45  tracking points  34  such as illustrated in  FIG. 18  for recognizing the face of the user from the image view illustrated in  FIG. 17 . The tracking points  34  of  FIG. 18  are an example, and may be adjusted in accordance with a processing capability of the control part and a resolution of the monitor  7 . 
     As mentioned above, by acquiring the tracking points  34  from the face of the user contained in the pre-makeup image, the makeup processing part  35  can set in a makeup parameter file such as illustrated in  FIG. 19  an application method of a makeup and colors by relating with the tracking points  34 . 
       FIG. 19  is a structure diagram of an example of a makeup process parameter file. In the makeup process parameter file of  FIG. 19 , an application method of a makeup and colors is set by being related with the tracking points  34  for each of an eye portion, a mouth portion, a cheek, etc. 
     The makeup process parameter file is set for each makeup pattern (image). The makeup process parameter file of  FIG. 19  represents an example of a class elegance. It should be noted that the above-mentioned four makeup patterns (images) are an example, and other makeup patterns (images) may be used. 
     Embodiment 1 
     (Lipstick Process) 
     The lipstick processing part  39  included in the simulation main application  28  performs a lipstick process by referring to the tracking points  34  of eight points in the lips and three points in the nose illustrated in  FIG. 20 .  FIG. 20  is an image view illustrating the tracking points referred to by the lipstick process. 
       FIG. 21  is a flowchart of an example illustrating the lipstick process. The lipstick process generally includes a preparation process of step S 10  and a color painting process of step S 20 . The preparation process includes a cutout and rotating process of step S 11 , a creating process of an image for extracting a contour of step S 12 , a contour extracting process of step S 13 , a spline curve creating process of step S 14 , a rotation returning process of step S 15  and a color painting map creating process of step S 16 . The color painting process includes a color painting process of step S 21  based on a color painting map and a debug and design drawing process of step S 22 . 
     Proceeding to the cutout and rotating process of step S 11 , the lipstick processing part  39  cutouts a part image  500  containing lips of the face of the user from the face recognition processing image, and rotates the part, image  500  to a position for processing to obtain a part image  501 . 
     Proceeding to the creating process of image for extracting a contour of step S 12 , the lipstick processing part  39  creates the image for extracting a contour from the part image  501 . Proceeding to the contour extracting process of step S 13 , the lipstick processing part  39  extracts a contour of the lips from the contour extracting image by points as illustrated in a part image  502 . 
       FIG. 22  is an image view of an example representing the contour extracting process. In a contour extracting image  600 , the tracking points of eight points in the lips and three points in the nose are superimposed. The lipstick processing part  39  extracts points of a contour extracting image  601  by re-searching based on the eight points in the lips and three points in the nose. 
       FIG. 23  is a screen comparing the tracking points of the eight points in the lips and the three points in the nose in the contour extracting image  600  with the points of contour extracting image  601  extracted by re-searching according to the eight points in the lips and the three points in the nose. It should be noted that if the above-mentioned re-searching is not performed well, default points such as illustrated in  FIG. 24 , which are calculated from the eight points in the lips and the three points in the nose in the contour extracting image  600 , may be used. 
       FIG. 24  is an image view of an example representing a process of acquiring the default points from the eight points in the lips and the three points in the nose. The lipstick processing part  39  acquires default points corresponding to five points in the upper lip by referring to the tracking points of the three points in the nose. 
     Proceeding to the spline curve creating process of step S 14 , the lipstick process part  39  completes a contour such as illustrated in  FIG. 25  by interpolating the points of the contour extracting image  601  or the default points such as illustrated in  FIG. 24  by a secondary spline.  FIG. 25  is an image view representing a process of completing a contour from the pints or the default points. After completion of the contour of the lips such as in an image picture  503 , the lipstick processing part  39  proceeds to step S 15 , and performs the rotation returning process to return the rotation performed by the cutout and rotating process of step S 11 . 
     Proceeding to the color painting map creating process of step S 16 , the lipstick processing part  39  creates a color painting map, which determines an intensity of color painting from brightness and color saturation of a part image  700 .  FIG. 26  is an image view of an example representing the color painting map creating process. Specifically, the lipstick processing part  39  crates a gray scale image  701  representing an intensity of color painting from the brightness and color saturation. The lipstick processing part  39  cutouts only a part image  702 , which is surrounded by the contour of the lips completed by the spline curve creating process, as a color painting map  504  from the gray scale image  701 . 
     Proceeding to the color paining process of step S 21  based on the color painting map, the lipstick processing part  39  performs coloring to the pre-makeup image according to the color painting map  504  created by the color painting map creating process of step S 16  and a makeup method and designated colors set in the makeup process parameter file such as illustrated in  FIG. 19 . 
       FIG. 27  is an image view of an example representing the color painting process based on the color painting map. The lipstick processing part  39  performs coloring to a pre-makeup image  803  to acquire a post-makeup image  804  in accordance with the method of applying a makeup and the designated colors set in the makeup process parameter file as illustrated in  FIG. 19  and the color painting map  802  created by the color painting map creating process of step S 16 . 
     Then, proceeding to the debug and design drawing process of step S 22 , the lipstick processing part  39  performs a process of design drawing, and, thereafter, the lipstick process is ended. 
     (Shadow Process) 
     The shadow processing part  38  included in the simulator main application  28  performs a shadow process by referring to the tracking points  34  of three points in the eye and one point in the eyebrow for each of left and right as illustrated in  FIG. 28 .  FIG. 28  is an image view illustrating the tracking points referred to by the shadow process. 
       FIG. 29  a flowchart of an example illustrating the shadow process. The shadow process generally includes a preparation process of step S 30  and a color painting process of step S 40 . The preparation process includes a creating process of a basic contour. 
     The color painting process includes a creating process of a color painting contour of step S 41 , which is repeated for the number of color painting patterns, a creating process of a color painting center of step S 42 , a creating process of a color painting map of step S 43 , a color painting process of step S 44  based on the color painting process, and a debug and design drawing process of step S 45 . 
     Proceeding to the creating process of a basic contour of step S 31 , the shadow processing part  38  obtains the form of the eye of the user such as illustrated in a part image  900  from the face recognition processing image.  FIG. 30  is an image view of an example representing the creating process of a basic contour. 
     The shadow processing part  38  performs two-point recognition of the contour of the eye (upper side border and lower side border), such as illustrated in a part image  1001 , by searching in upward and downward directions from the center the eye in order to create a contour used as a base of the color painting contour. The shadow processing part  38  adds the four points created by the interpolation to the four points including the two points of the recognized contour of the eye and the outer corner and inner corner of the eye, and creates a polygon by a total of eight points as illustrated in a part image  1002 . 
     Proceeding to the creating process of a color painting contour of step S 41 , the shadow processing part  38  creates a color painting contour like a partial picture  901 .  FIG. 31  is an image view of an example illustrating the creating process of a color painting contour. In order to create a color painting contour, as illustrated in a partial image  1101 , the shadow processing part  38  creates a color painting contour on the basis of outer corner and inner corner of eye. It should be noted that an expansion of a contour, a shift of a top point, and the like may be done by a parameter designation by GUI. 
     Proceeding to the creating process of a color painting center of step S 42 , the shadow processing part  38  creates a position of a color painting center like () illustrated in a partial image  902 . Proceeding to the color painting map creating process of step S 43 , the shadow processing part  38  creates a color painting map, which determines an intensity of color painting like a partial image  903 . 
     Specifically, the color painting map creating process determines an intensity of painting corresponding to a distance from the color painting center to a side of the polygon. For example, the shadow processing part  38  determines the color painting map so that an intensity of color painting is decreased as it is closer to a side. It should be noted that the color painting map creating process is aimed at a portion excluding the basic contour from the color painting contour. 
     The shadow processing part  38  creates a further smoother gradation by applying a gradation process to the created color painting map as illustrated in  FIG. 32 .  FIG. 32  is an image view of an example illustrating a color painting map which has not been subjected to the gradation process and a color painting map which has been subjected to the gradation process. 
     Proceeding to the color painting process of step S 44  based on the color painting map, the shadow processing part  38  performs coloring on the pre-makeup image to obtain the post-makeup image in accordance with the color painting map  903  created by the color painting map creating process of step S 43  and a makeup applying method and designated colors set in the makeup process parameter file such as illustrated in  FIG. 19 . 
     Then, proceeding to the debug and design drawing process of step S 45 , the shadow processing part  38  performs a debug and design drawing process, and, thereafter, ends the shadow process. It should be noted that the shadow processing part  38  may realize a multi-color painting by repeating the process of steps S 41 -S 45  for the number of color painting patterns. 
     (Cheek Process) 
     The cheek processing part  40  included in the simulator main application  28  performs a cheek process by referring to the tracking points  34  of an outer corner of eye and a corner of mouth (separately on left and right) and a middle of eyes and a nose center (for stabilizing) such as illustrated in  FIG. 33 .  FIG. 33  is an image view illustrating tracking points referred to be the cheek process. 
       FIG. 34  is a flowchart of an example illustrating the cheek process. The cheek process includes a creating process of a color painting contour of step S 50 , a color painting process of step S 51 , and a debug and design drawing process of step S 52 . 
     Proceeding to the color painting contour creating process of step S 50 , the cheek processing part  40  creates a contour polygon as a color painting contour on the basis of an outer corner of eye and a corner of mouth in order to create a color painting contour.  FIG. 35  is an image view of an example illustrating the color painting contour. It should be noted that a number of points, a size, a shape or a position of the contour polygon may be parameter-designated by GUI. 
     Proceeding to the color painting process of step S 51 , the cheek processing part  40  determines an intensity of painting corresponding to a distance from a color painting center to a side of the contour polygon. It should be noted that if it requires an excessive cost in determining an intensity of painting, the intensity of painting may be determined by decreasing (thinning like a mosaic pattern) the resolution (parameter-designated by GUI). The cheek processing part  40  performs coloring on the pre-makeup image to obtain the post-makeup image in accordance with the determined intensity of painting and the makeup applying method and designated colors set in the makeup process parameter file such as illustrated in  FIG. 19 . 
     Then, proceeding to the debug and design drawing process of step S 52 , the cheek processing part  40  performs the debug and design drawing process, and, thereafter, ends the cheek process. 
     (Eyebrow Process) 
     The eyebrow processing part  37  included in the simulator main application  28  performs the eyebrow process, separately on left and right, by referring to the tracking points  34  of an outer corner of eye, an eye center and tow points in an eyebrow as illustrated in  FIG. 36 .  FIG. 36  is an image view illustrating the tracking points referred by the eyebrow process. 
       FIG. 37  is a flowchart of an example illustrating the eyebrow process. Generally, the eyebrow process includes an eyebrow contour extracting process of step S 60 , an erasing process of an original eyebrow area of step S 70 , a deforming process of an eyebrow form of step S 80 , and a pasting process of a deformed eyebrow of step S 90 . 
     The erasing process of an original eyebrow area includes an area expanding process of step S 71  and an eyebrow erasing process of step S 72 . The deforming process of an eyebrow form includes a creating process of step S 81  of a designation curve corresponding to a deformation designation and a deformed eyebrow creating process of step S 82 . 
     Proceeding to the eyebrow contour extracting process of step S 69 , the eyebrow processing part  37  acquires a form of an eyebrow of the face of the user such as in a partial image  2001  from the face recognition processing image.  FIG. 38  is an image view of an example illustrating the eyebrow contour extracting process. As illustrated in a partial image  2101 , the eyebrow processing part  37  searches the eyebrow rightward and leftward with the outer side of the eye brow (actually, heat the center of eyebrow) as a center in order to extract the contour of the eyebrow. The eyebrow processing part  37  recognizes a contour form of the eyebrow from a search result, as illustrated in a partial image  2102 . 
     Proceeding to the area expanding process of step S 71 , the eyebrow processing part  37  expands the area representing the contour form of the recognized eyebrow. Proceeding to the eyebrow erasing process of step S 72 , the eyebrow processing part  37  erases the eyebrow by over-painting the expanded area with a skin color of the vicinity. Additionally, the eyebrow processing part  37  apply a fitting process to a border part of the expanded area. 
     Proceeding to the creating process of step S 81  of a designation curve corresponding to a deformation designation, the eyebrow processing part  37  deforms the area (skeleton) representing the contour form of the eyebrow.  FIG. 39  is an image view of an example illustrating the creating process of a designation curve corresponding to a deformation designation. 
     As illustrated in a partial image  2201 , the eyebrow processing part  37  can apply a deforming process like, for example, a skeleton  2203 , by replacing an area representing the contour form of the recognized eyebrow by a skeleton  2002  formed by a plurality of strips extending in vertical direction to the axis line in a horizontal direction and changing the form of the axis line and heights of the strips. 
     Proceeding to the deformed eyebrow creating process of step S 82 , the eyebrow processing part  37  creates a deformed eyebrow from the skeleton  2203 . Proceeding to the deformed eyebrow pasting process of step S 90 , the eyebrow processing part  37  pastes the deformed eyebrow to the pre-makeup image to obtain the post-makeup image. 
     (Foundation Process) 
     The foundation processing part  36  included in the simulator main application  28  performs the foundation process, separately on left and right, by referring to the tracking points  34  of an outer corner and an inner corner of the eye, one point in the eyebrow, a middle of the eyes, and a nose center as illustrated in  FIG. 40 .  FIG. 40  is an image view illustrating the tracking points referred to by the foundation process. 
       FIG. 41  is a flowchart of an example illustrating the foundation process. The foundation process includes a creating process of a contour of step S 101 , a gradation process of step S 102  to an objective image, and an image pasting process of step S 103 . It should be noted that the process of steps S 101 -S 103  is repeated for a number of objective areas. 
     Proceeding to the contour creating process of step S 101 , the foundation processing part  36  creates three kinds of contours (four locations) of a forehead, a nose and cheeks (left and right) as illustrated in  FIG. 42 .  FIG. 42  is an image view of an example illustrating the contours. It should be noted that a size, a position, etc., of each contour may be parameter-designated by GUI. 
     Proceeding to the gradation process of step S 102  to the objective image, the foundation process part  36  performs a gradation process on the objective images corresponding to the created contours as illustrated in  FIG. 43 .  FIG. 43  is an image view of an example illustrating an original image before the gradation process and an original image after the gradation process. According to the gradation process on the objective image, the foundation processing part  36  can perform smoothing small roughness of skin. 
     Proceeding to the image pasting process of step S 103 , the foundation processing part  36  pastes the objective images after the gradation process to the three-kinds of contours (four locations) including the forehead, the nose and cheeks (left and right) in the pre-makeup image to obtain the post-makeup image. 
     Embodiment 2 
     In the embodiment 2, descriptions will be given of the lipstick process, the shadow process, the cheek process, the eyebrow process and the foundation process and other examples. 
     (Lipstick Process) 
     The lipstick processing part  39  included in the simulator main application  28  performs the lipstick process by referring to the tracking points  34  as mentioned below. 
       FIG. 44  and  FIG. 45  are flowcharts of an embodiment of the lipstick process (lip pen drawing process). In  FIG. 45 , first in step S 301 , a face image of a rectangle area surrounding a mouth is acquired, and also contour data of the mouth is acquired. The contour data of the mouth is obtained according to the tracking points (feature points)  34   
     In next step S 302 , a curve drawing section and a straight line drawing section are set according to a makeup pattern selected by a user.  FIG. 46  is an image view of an example of the face image of the rectangle area surrounding the mouth. 
     In this figure, besides each feature points of upper ends M 2  and M 4  of the mouth (upper lip), right and left ends M 1  and M 5  of the mouth, and a lower end M 6  of the mouth (lower lip), there are indicated a point M 7  at a distance of ⅓ of a mouth width (distance between M 1  and M 5 ) from the left end M 1  at the lower edge of the mouth (lower lip) and a pint M 8  at a distance of ⅓ of the mouth width (distance between M 1  and M 5 ) from the right end M 5  at the lower edge of the mouth (lower lip). 
     If the selected makeup pattern is fresh, the lipstick processing part  39  draws each of a section M 1 -M 2  and a section M 4 -M 5  by a curve, and draws each of a section M 2 -M 3  and a section M 3 -M 4  by a straight line. Thereby, as illustrated in a drawing image of  FIG. 47(A) , a lip pen drawing, which gives roundness to the upper lip, is performed, and no lip pen drawing is performed on the lower lip. 
     If the selected makeup pattern is sweet, the lipstick processing part  39  draws each of the section M 1 -M 2 , the section M 4 -M 5  and a section M 1 -M 6 -M 5  by a curve, and draws each of the section M 2 -M 3  and the section M 3 -M 4  by a straight line. Thereby, as illustrated in a drawing image of  FIG. 47(B) , a lip pen drawing, which gives roundness to the upper lip and the lower lip, is performed. 
     If the selected makeup pattern is cute, the lipstick processing part  39  draws each of a section M 1 -M 7 , a section M 7 -M 8  and a section M 8 -M 5  by a straight line. Thereby, as illustrated in a drawing image of  FIG. 47(C) , a lip pen drawing, which makes the lower lip linear, is performed and no lip pen drawing is performed on the upper lip. 
     If the selected makeup pattern is cool, the lipstick processing part  39  draws a section M 2 -M 4  by a curve, and draws each of the section M 1 -M 2 , the section M 4 -M 5 , a section M 1 -M 7 , the section M 7 -M 8  and the section M 8 -M 5  by a straight line. Thereby, as illustrated in a drawing image of  FIG. 47(D) , a lip pen drawing, which makes the upper lip and the lower lip linear, is performed. 
     Subsequently, the lipstick processing part  39  performs a loop process of step S 306 . Here, the lipstick processing part  39  performs the following process by sequentially increasing by a pixel unit from a y-coordinate value of the contour data of the mouth+an offset value−βto the y-coordinate of the contour data of the mouth+the offset value+β. β is ½ of a pen width of the lip pen (for example, several millimeters). 
     In step S 307  in the loop of step S 306 , the lipstick processing part  39  computes each of color saturation (S), hue (H) and brightness (V) from RGB values of a pixel corresponding to the xy-coordinates by using a predetermined formula. 
     Then, in step S 308 , the lipstick processing part  39  applies a gradation by correcting the HSV values of the color of the lip pen to be thinner than a difference (the offset value+β at maximum) between the y-coordinated of the contour data of the mouth and the y-coordinate, and, thereafter, adds it to the HSV values of the pixel corresponding to the xy-coordinates obtained in step S 307 . 
     Thereafter, the lipstick processing part  39  converts, in step S 309 , the HSV values of the pixel corresponding to the above-mentioned xy-coordinates concerned into RGB values, and performs, in step S 310 , a lip pen drawing by overwriting and updating the pixel corresponding to the xy-coordinates concerned by using the RGB values. 
     It should be noted that although step S 306  is performed only once because the upper edge of the upper lip is the only curve drawing section in the drawing image of  FIG. 47(A) , it is performed twice in the drawing image of  FIG. 47(B)  because the upper edge and the lower edge of the upper lip are curve drawing sections in the drawing image of  FIG. 47(B) . When drawing the lower edge of the lower lip, the offset value of  FIG. 48  is handled as a negative value. After completing the loop process of step S 303 , the lipstick processing part  39  proceeds to step S 311  to acquire a function of the straight line of each straight line drawing section. In the loop process of the subsequent step S 312 , the lipstick processing part  39  sequentially increase the x-coordinate value from 0 (the position of the left end M 1  of the mouth) to the maximum value (the position of the right end M 5  of the mouth) by a pixel unit. 
     The lipstick processing part  39  determines, in step S 313  in the loop, whether or not it is a straight line drawing section, and if it is not a straight line drawing section, returns to step S 312  to increase the x-coordinate value. If it is a straight line drawing section, proceeding to step S 314  where the lipstick processing part  39  acquires the y-coordinate Y(L) of the straight line corresponding to the x-coordinate concerned by using the function of the straight line. 
     Next, the lipstick processing part  39  performs a loop process of step S 315 . Here, the lipstick processing part  39  performs the following process by sequentially increasing the y-coordinate value from Y(L)−β to Y(L)+β by a pixel unit. β is ½ of a pen width of the lip pen (for example, several millimeters). 
     In step S 316  in the loop of step S 315 , the lipstick processing part  39  computes each of color saturation (S), hue (H) and brightness (V) from RGB values of a pixel corresponding to the xy-coordinates by using a predetermined formula. Then, in step S 317 , the lipstick processing part  39  applies a gradation by correcting the HSV values of the color of the lip pen to be thinner than a difference (beta at maximum) between Y(L) and the y-coordinated, and, thereafter, adds it to the HSV values of the pixel corresponding to the xy-coordinates obtained in step S 316 . 
     Thereafter, the lipstick processing part  39  converts, in step S 318 , the HSV values of the pixel corresponding to the above-mentioned xy-coordinates concerned into RGB values, and performs, in step S 319 , a lip pen drawing by overwriting and updating the pixel corresponding to the xy-coordinates concerned by using the RGB values. 
     It should be noted that although step S 315  is performed only once because the upper edge of the upper lip is the only curve drawing section in the drawing image of  FIG. 47(C) , it is performed twice in the drawing image of  FIG. 47(D)  because the upper edge and the lower edge of the upper lip are curve drawing sections in the drawing image of  FIG. 47(D) . After completing the loop process of step S 312 , the lipstick processing part  39  ends the lipstick process. 
     As mentioned above, a desired type is selected from a plurality of makeup patterns, the lipstick processing part  39  sets a form to draw (color painting process) by a lip pen in accordance with the selected makeup pattern, and performs the drawing by the lip pen with the set form, and, thereby, the vicinity of the contour of the mouth can be drawn and displayed in accordance with the selected makeup pattern by merely selecting a desired type. 
     Moreover, because the lipstick processing part  39  applies a gradation to dilute the color of the lip pen as a vertical direction distance from a contour of a mouth increases, the drawn color of the lip pen fits to the skin color, which permits displaying the color of the lip pen providing no uncomfortable feeling. The lipstick processing part  39  included in the simulator main application  28  may compute the following points from the tracking points  34  to perform the lipstick process of embodiment 1 or 2. 
     Positions of a plurality of points previously set up for grasping morphological features of a mouth can be computed from the tracking points  34 .  FIG. 49  is an image view of an example illustrating positions of a plurality of points previously set up for grasping morphological features of a mouth.  FIG. 49  illustrates the following 14 points. 
     The 14 points illustrated in  FIG. 49  are a total of 14 points including positions (two points) of both sides of a nose, center positions (two points) of nostrils, positions (two points) of a peak of an upper lip, a position of a trough of the upper lip, a center position of the lip, positions (two points) of corners of a mouth, a position of a center of a lower edge of the lower lip, a center portion of a jaw, and positions (two points) of both sides of the jaw. It should be noted that the number and the positions of those points are not limited, and the number and the positions can be changed if morphological features of a mouth can be grasped and if it is suitable for creating makeup information. 
     The lipstick processing part  39  determines a planer feature analyzed according to the positions of the 14 points with respect to the following five items, and grasps a balance of an entire mouth of an object person. The lipstick processing part  39  compares the grasped balance of the entire mouth of the object person with a most appropriate balance as a reference so as to measure a difference therebetween, and corrects portions different from the reference. It should be noted that the balance of lips set as a reference depends on a proportion of lips evaluated as beautiful. 
     The five items to determine a balance of a form of a mouth are positions of corners of the mouth, a position of an upper lip, a position of a lower lip, a position of a peak of the upper lip, and angles of the peak and trough of the upper lip. References for the five items in the most suitable balance are: for positions of a mouth, a position moving an inner side of a black eye; for a position of an upper lip, a position at ⅓ of a distance from under a nose to a center position of lips; for a position of a lower lip, a position at ⅓ of a distance from a center position of a jaw to a center position of the lips; for a position of a peak of the upper lip, a position moving the a middle of nasal apertures downward; and for angles of the peak and trough of the upper lip, an angle decreased by 10 degrees from the peak toward the trough. 
     According to the optimum reference balance of lips, the lipstick processing part  39  compares the balance of the person to be applied with a makeup to grasp a difference therebetween, and corrects the object lips to the reference balance. Here, a technique of the correction is explained. First, the lipstick processing part  39  draws a horizontal line from the center of the lips and measures whether the positions of the corners of the mouth are higher or lower than the horizontal line. If the positions of the corners of the mouth are higher than the horizontal line, the lipstick processing part  39  does not perform a correction. If the positions of the corners of the mouth are lower than the horizontal line, the lips are seen as slack and loose, and, thus, the lipstick processing part  39  applies a correction makeup to move the positions of the corners of the mouth upward to the extent of 2 mm. 
     It should be noted that the reason for setting the limit of adjustment to 2 mm is to avoid a result of the adjustment from being unnatural. For example, it is usual to change it within a range of about 2 mm in a case where a beauty consultant gives an advice for a makeup of lips to a client at a shop front and makes a suggestion of a makeup method to approach to standard lips. As mentioned above, if the correction range exceeds 2 mm, it is not preferable because the makeup may provide unnaturalness. It should be noted that in a case where the original points of the corners of the mouth are offset, the points of the corners of the mouth are adjusted manually because the points in the optimum balance are also offset. The range of the correction, that is 2 mm, is the same in other portions. 
     Next, the lipstick processing part  39  corrects the form of the peak and trough of the upper lip. The lipstick processing part  39  sets up a position of the peak of the upper lip by determining based on a reference of a position, which is at ⅓ of a distance from the bottom of the nose to the center position of the lips and a position moved downward from the middle of the nasal apertures, and sets up a makeup point on a screen so that the peak of the upper lip comes to the thus-determined point. 
     Next, the lipstick processing part  39  sets up a position of the lower lip to a position at ⅓ of a distance from a center line of the lips to the center position of the jaw, and, further, draws a line of the lower jaw by connecting the center position and three point positions on both sides thereof by a circular arc and draws a line of the lower lip in a similar form as the circular arc form of the lower jaw. The drawing of the lower jaw can be done automatically by deforming a basic form along the line of the lower jaw on a computer screen. 
     As mentioned above, the lipstick processing part  39  compares the balance of the entire lips of the object person with the optimum balance in accordance with the references of the five items to grasp differences thereof, and can acquire a line for correcting it to the optimum balance. 
     (Shadow Process) 
     The shadow processing part  38  included in the simulator main application  28  performs the shadow process by referring to the tracking points  34  as follows. It should be noted that the shadow process is performed in an order of an eyeline searching process, an eyeline drawing process, and an eye shadow applying process. 
       FIG. 50  is a flowchart of an embodiment of the eyeline searching process. In the figure, first, in step S 401 , a face image of a rectangle area surrounding an eye and an eyebrow is acquired, and, in step S 402 , an area for retaining a binary image is reserved in the memory device or the like. A threshold value TH is sequentially increased by a predetermined increment from a lowest value to a maximum value in a loop process of the subsequent step S 401 , and binarization of the face image of the rectangle area surrounding the eye and the eyebrow is performed using the threshold value TH in step S 404  in the loop. Performing the binarization by varying the threshold value TH is to accurately detect an eyeline. 
     In the subsequent step S 405 , a median filter process is performed on an edge image obtained by the binarization to eliminate noise. This is performed to eliminate a noise generated due to an eyelash or the like. Then, it is determined, in step S 406 , whether or not there are a plurality of edge pixels continuously extending in a direction of a width of the eye, that is, whether there are contour line forming pixels. If the above-mentioned contour line forming pixels do not exist, the threshold value TH is increased by a predetermined increment, and it returns to step S 404 . 
     In step S 407 , a contour line is extracted from the above-mentioned binary image. Then, the shadow processing part  38  linear-interpolates (or curve-interpolates) a discontinuous portion in the contour line extracted from the above-mentioned binary image, and, further performs, in step S 409 , a median filter process on the interpolated contour line to eliminate a noise, and ends the process. 
       FIG. 51  is a flowchart of an embodiment of the eyeline drawing process. This process is perforated on an area of an upper side (eyebrow side) of the contour line of the upper eyelid connecting inner corners of eye EL 4 , ER 4  and outer corners of eye EL 1  and ER  1 . 
     In the figure, first, in step S 410 , an eyeline drawing pattern is set. In a case where cool is selected as a makeup pattern, the eyeline drawing pattern is an eyeline of a width from an inner corner of eye to an outer corner of eye of the upper eyelid, and an eyeline drawing pattern is set to perform a gradation of the eyeline on the outer corner side in the x-direction in a hutched area Ia. 
     Moreover, in a case where fresh is selected as a makeup pattern, as illustrated in  FIG. 53 , the eyeline drawing pattern is an eyeline of a width from a middle of the upper eyelid (a middle point between the inner corner and the outer corner) to the inner corner of eye, and an eyeline drawing pattern is set to perform a gradation of the eyeline on the outer corner side in the x-direction in a hutched area Ib. 
     Moreover, in a case where sweet is selected as a makeup pattern, as illustrated in  FIG. 54 , the eyeline drawing pattern is an eyeline of a width from the inner corner of eye to the outer corner of eye of the upper eyelid, and an eyeline drawing pattern is set to not perform a gradation of the eyeline. 
     Further, in a case where cute is selected as a makeup pattern, as illustrated in  FIG. 55 , the eyeline drawing pattern is an eyeline of a width from the inner corner of eye to the outer corner of eye of the upper eyelid, and an eyeline drawing pattern is set to perform a gradation of the eyeline on the eyebrow side in the y-direction in a hutched area Id. 
     The eyeline drawing patterns corresponding to the above-mentioned makeup patterns are default values when an eyeline selection box is not touched, and it is possible to select a desired pattern from among 4 eyeline drawing patterns displayed when the eyeline selection box is touched. 
     Subsequently, an x-coordinate value is sequentially increased by a loop process from 0 (a position of the inner corner of eye) to a maximum value (a position of the outer corner of eye) by a pixel unit. The loop process of step S 412  is performed for each x-coordinate value within this loop. Here, the y-coordinate value is increased from 0 (y-coordinate of the contour line) to a maximum value (eye level width: a maximum separation distance between the upper eyelid and the lower eyelid) to perform the following process. 
     Brightness of the pixel designated by the above-mentioned x-coordinate and y-coordinate is calculated in step S 413 , and it is determined, in step S 411 , whether or not the brightness of the pixel is equal to the brightness of the eyeline. Here, if they are not equal to each other, the y-coordinate value is increased, and it returns to step S 413 . 
     If the bright of the pixel is equal to the brightness of the eyeline, it proceeds to step S 415  to decrease the brightness of the pixel from a current brightness by a predetermined value. Thereby, the brightness of the pixel on the contour line is decreased and it is deepened, which results in highlighting the eyeline. Thereafter, it is determined, in step S 416 , whether it is the area Ia of  FIG. 52  (a case of sweet) or the area Ib of  FIG. 53  (a case of cool) according to the x-coordinate and the y-coordinate, and if it is a case of the above-mentioned areas Ia or Ib, a gradation of the eyeline is performed, in step S 417 , on the side of the outer corner of eye in the x-direction. 
     Further, it is determined, in step S 418 , whether it is the area Id of  FIG. 55  (a case of fresh) according to the x-coordinate and the y-coordinate, and if it is a case of the above-mentioned areas Ia and Ib, a gradation of the eyeline is performed, in step S 419 , on the side of the eyebrow in the y-direction. Thereafter, after the loop processes of the above-mentioned steps S 411  and S 412 , the shadow processing part  38  ends the eyeline searching process. 
       FIG. 56  is a flowchart of an embodiment of the eye shadow application process. In the figure, first, in step S 430 , an eye shadow application area is set. Here, a maximum distance Ymax in the y-direction from the contour line of the upper eyelid to the contour line of the eyebrow is acquired, and a maximum application size My=Ymax/α is acquired. It should be noted that if the makeup patter is sweet or cute, α-2, if cool, α=3, and if fresh, α=6. 
     Further, an upper limit (eyebrow side) of the eye shadow application area is acquired for each makeup pattern using curves illustrated in  FIG. 58(A) , (B). At this time, the above-mentioned maximum application size My is used. Here, if it is a case of sweet, cool or cute, a curve from the inner corner of eye to the outer corner of eye illustrated in  FIG. 58(A)  is used, and if it is a case of fresh, a curve from a point on the contour line of the upper eyelid separate from the inner corner of eye by a predetermined length to the inner corner of eye as illustrated in  FIG. 58(B)  is used. Additionally, a lower limit of the eye shadow application area is set to the contour line of the upper eyelid. 
     Moreover, the start point of the movement locus is set in step S 431 , and an application size at the start point of the eye shadow application is set in step S 432 . Although  FIG. 59  illustrates an eye shadow application area corresponding to  FIG. 58(A) , a roughly center position P 0  of the lower limit (the contour line of the upper eyelid) of the eye shadow application area is set to the start point of the eye shadow application. Additionally, a radius of a circle having a center as the start point and contacting the upper limit of the eye shadow application area is set to the application size, and, further, arrows A 1  and A 2  illustrated in overlapping with the lower limit of the eye shadow application area are set to the movement locus of the start point. 
     Next, in step S 433 , a density of eye shadow at the start point is computed by using a predetermined formula from the skin color of the user and the color of the selected eye shadow, and the eye shadow of the obtained color is applied to the start point. Then, an air brush process is performed, in step S 434 , to make the color density of the eye shadow applied to the start point thinner (gradate) in proportion to a distance from the start point in order to acquire a color of each pixel at which the eye shadow covers the skin by adding the thus-obtained density of color of the eye shadow of each pixel position to the skin color of the pixel at the same position, thereby updating the color of each pixel. 
     It should be noted that, in the above-mentioned air brush process, only inside the circle having a radius corresponding to the application size in the eye shadow application area is an object to be processed, and no process is applied to the lower half circle portion under the lower limit of the eye shadow application area. Additionally, as a relationship between a distance from the center in the air brush process and a density, a property that the density is decreased in proportion to a distance from the center such as illustrated in  FIG. 60(A)  is used in a case where the makeup pattern is cute or fresh, and a property that a degree of decrease in the density is larger when the distance from the center is small and a degree of decrease in the density is smaller when the distance from the center is large such as illustrated in  FIG. 60(B)  is used in a case of sweet or cool. 
     Thereafter, in step S 435 , the start point is moved by a predetermined distance in accordance with the movement locus of the start point indicated by the arrows A 1  and A 2 . It should be noted that the movement of the start point from the position P 0  in the direction of the arrow A 1  by a predetermined distance is repeated, and if the moved start point is out of the eye shadow application area, it returns to the position P 0  and the start point is moved in the direction of the arrow A 2  by the predetermined distance. 
     Moreover, a new application size is calculated in step S 436 . The new application size decreases at a rate of several percent to several tens percent as the start point moves from the position P 0  in the directions of the arrows A 1  and A 2 . 
     Subsequently, it is determined, in step S 437 , whether or not it is an end point of the eye shadow application, and if it is not the end point, the above-mentioned steps S 433  to S 436  are repeated, and if it is an end point, the shadow processing part  38  ends the eye shadow applying process. The determination of the end point is such that the eye shadow application is ended when the start point moves in the direction of the arrow A 2  and out of the eye shadow application area. 
     As mentioned above, a desired type is selected from a plurality of makeup patterns, an area for applying the eye shadow is set in an eye portion of the face image in accordance with the selected makeup pattern, and the eye shadow is applied by overlapping the color of the eye shadow over the color of the face image in the area where the eye shadow is applied, and, therefore, the eye shadow can be applied and displayed in the eye portion of the face image in accordance with the selected makeup pattern by merely selecting the desired type. 
     Moreover, because the contour of eye is detected and the detected contour of the eye and the vicinity thereof is drawn in accordance with the selected makeup pattern, the eyeline can be drawn and displayed in the eye portion of the face image in accordance with the selected makeup pattern by merely selecting a desired type. 
     Moreover because a gradation is applied by decreasing the density of the eye shadow as it apart from the start point of the application in the area where the eye shadow is applied, the applied eye shadow fits with the skin color of the face image and the eye shadow having no feeling of strangeness can be displayed, and because the start point is sequentially moved in the area where the eye shadow is applied, the applied eye shadow fits with the skin color of the face image and the eye shadow having no feeling of strangeness can be displayed irrespective of a form of the area where the eye shadow is applied in the face image. 
     The shadow processing part  38  included in the simulator main application  28  may be configured to perform an eye makeup to exhibit eyes larger and attractive by well-balanced eye by comparing a classified and grasped form of eye of the object person for makeup with a form of a standard-balanced eye to approximate the standard-balanced eye. 
     The morphological features of eye can be classified using four elements as indexes including a frame axis indicating a form of a lid fissure, a form axis indicating convexoconcave form of an eye, an angle axis indicating angle form of an eye, and a form of a standard-balanced eye. The frame form is a contour shape of a lid fissure formed by upper and lower eyelid using an eyelash line as a target. The frame axis is an arrangement on an axis according to a ratio of a top-to-bottom diameter and a left-to-right diameter. A frame axis is provided, for example, as a vertical axis. The form of the standard-balanced eye is arranged at a center of the frame axis. A form of an eye having a longer top-to-bottom diameter and a short left-to-right diameter with a ratio of the top and bottom diameter of the lid fissure and the left-to-right diameter of the lid fissure is 1:3 is arranged on one side, that is, upper side of the frame axis. A form of an eye having a short top-to-bottom diameter and a long left-to-right diameter is arranged on the other side, that is, the lower side of the axis. 
     Moreover, the form indicating a convexoconcave shape of an eye is grasped by, for example, a convexoconcave shape of a lid fissure and prominences of upper and lower eyelids. The form axis is configured as a horizontal direction axis perpendicular to the above-mentioned frame axis. The form of the standard-balanced eye is arranged at the center of the form axis. On one side, that is, the left side of the form axis, there is arranged a form of an eye in which prominences of an upper eyelid is more planar than the form of standard-balanced eye (a prominence form of a full-fleshed eyelid which is general in a single-edged eyelid or a concealed double-edged eyelid, a fresh of the lower eyelid is thin, and a curve of an eyeball is inconspicuous. On the other side, that is, the right side of the axis, there is arranged a form of an eye in which a prominence of the upper eyelid is stereoscopic (a sharply-chiseled state generally seen in double-edged eyelid and triple-edged eyelid, there is a depression on a border between an eyebrow arch bone and an orbit, and there is seen remarkable a prominence of an eyeball), and a noticeable curved surface of an eyeball appears in the lower eyelid, or the lower eyelid is stereoscopic due to tumor of orbital fat. 
     Moreover, the angle form of an eye is an angle formed by a horizontal line passing the inner corner of eye and a diagonal line connecting the inner corner and the outer corner of eye. The angle form of the standard-balanced eye is larger than 9 degrees and smaller than 11 degrees, most preferably 10 degrees. The shadow processing part  38  determines that if it is larger than 9 degrees and smaller than 11 degrees, it is standard, if equal to or smaller than 9 degrees, outer corner drop, and is equal to or larger than 11 degrees, outer eye raised. The angle axis indicating up and down of the angle form of eye is represented as it individually exists in four quadrants defined by two axes when projecting the above-mentioned frame axis and form axis onto a flat surface. 
     The feature of the standard-balanced eye is that, as illustrated in  FIG. 61 , a) it has a frame form in which a ratio of the top-to-bottom diameter of eye to the left-to-right diameter of eye is a frame form of 1; 3; b) there is no remarkable irregularity in the form of the upper and lower eyelids when viewing the face from a side surface and has a curve from an eyebrow arch bone (a slightly lifted bone under a portion where the eyebrow are grown) to a cheek bone is smooth; c) an angle formed by a horizontal line passing the inner corner of eye and a line connecting an inner corner; d) an outer corner of eye is 10 degrees, and d) a lid groove has an intermediate form between a double-edged eyelid and a concealed double edged eyelid, and the groove on the inner corner side is narrow and the groove on the outer corner side is wider than that of the side; and e) a width of the top-to-bottom diameter and a width from the upper edge of the lid fissure to the eyebrow has a form of balance of 1:1  FIG. 61  is a front view illustrating the form of the standard-balanced eye. 
     (Cheek Process) 
     The cheek processing part  40  included in the simulator main application  28  performs a cheek process by referring to the tracking points  34  as mentioned below. 
       FIG. 62  is a flowchart of an embodiment of a rouge application process (cheek process). In the figure, first, an area of applying a rouge and a start point are set in step S 501 , a movement locus of the start point is set in step S 402 , and an application size at the start point of the rouge application is set in step S 503 . The processes of the above-mentioned steps S 501 -S 503  differ depending on a makeup pattern selected by a user. 
     In a case where the makeup pattern is sweet, as illustrated in  FIG. 63 , if, within the face area, a y-coordinate of feature points of outer corners of eye EL 1  and ER 1  is set to y 1 ; a y-coordinate of feature points of left and right ends of nose N 1  and N 3  is set to y 2 ; a y-coordinate of feature points of left and right ends of mouth M 1  and M 5  is set to y 3 ; an x-coordinate of feature points of inner corners of eye EL 4  and ER 4  is set to x 1 ; and an x-coordinate of feature points of outer corners of eye EL 1  and ER 1  is set to x 3 , the rouge application area is set to a range of which y-coordinate is (y 1 +y 2 )/2 to (y 2 +y 3 )/2 and x-coordinate is between x 1  and x 3 , and a roughly center position P 0  within the area concerned is set to a start point of the rouge application. Additionally, ½ of a vertical width of the area is set to an application size, and, further, the arrows A 1  and A 2  are set as a movement locus of the start point. 
     In a case where the makeup pattern is cool, as illustrated in  FIG. 64 , if, within the face area, a y-coordinate of feature points of outer corners of eye EL 1  and ER 1  is set to y 1 ; a y-coordinate of feature points of left and right ends of mouth M 1  and M 5  is set to y 3 ; an x-coordinate of feature points of pupils PL and PR is set to x 2 ; an x-coordinate of feature points of outer corners of eye EL 1  and ER 1  is set to x 2 ; and an x-coordinate of feature points F 6  and F 7  on the contour line of the face and having the same y-coordinate as the inner corners of eye EL 1  and ER  1  is set to x 4 , the rouge application area is set to a range encircled by a line connecting a point (x 2 , y 3 ) and a point (x 4 , y 1 ), a line of y=y 3 , and the contour of the face, and a position P 0  within the area concerned is set to a start point. Additionally, a radius of a circle having a center at the position P 0  and contacting the rouge application area is set as an application size, and the arrows A 1  and A 2  are set as a movement locus of the start point. 
     In a case where the makeup pattern is cute, as illustrated in  FIG. 65 , if, within the face area, a y-coordinate of feature points of outer corners of eye EL 1  and ER 1  is set to y 1 ; a y-coordinate of feature points of left and right ends of nose N 1  and N 3  is set to y 2 ; a y-coordinate of feature points of left and right ends of mouth M 1  and M 5  is set to y 3 ; an x-coordinate of feature points of pupils PL and PR is set to x 2 ; an x-coordinate of feature points F 6  and F 7  on the contour line of the face and having the same y-coordinate as the inner corners of eye EL 1  and ER  1  is set to x 4 ; and an x-coordinate of feature points F 10  and F 11  on the contour line of the face and having the same y-coordinate as the ends of mouth M 1  and M 5  is set to x 5 , the rouge application area is set to a range encircled by a line connecting a point (x 4 , y 1 ) and a point (x 2 , y 2 ), a line connecting a point (x 2 , y 2 ) and a point (x 5 , y 3 ) and the contour of the face, and a position P 0  within the area concerned is set to a start point. Additionally, a radius of a circle having a center at the position P 0  and contacting the rouge application are is set as an application size, and the arrows A 1  and A 2  are set as a movement locus of the start point. 
     In a case where the makeup pattern is fresh, as illustrated in  FIG. 66 , if, within the face area, a y-coordinate of feature points of outer corners of eye EL 1  and ER 1  is set to y 1 ; a y-coordinate of feature points of left and right ends of mouth M 1  and M 5  is set to y 3 ; and an x-coordinate of feature points of inner corners of eye EL 4  and ER 4  is set to x 1 , the rouge application area is set to a range of which y-coordinate is from y 3  to (y 1 +y 3 )/2 and x-coordinate is from x 1  to the contour of the face, and the position P 0  within the area concerned is set to a start point of the rouge application. Additionally, ½ of a vertical width of the area is set to an application size, and, further, the arrows A 1  and A 2  are set as a movement locus of the start point. 
     Next, an application of rouge is performed. Here, first in step S 504 , a concentration of rouge at the start point is computed by using a predetermined formula from the skin color of the user and the color of the selected rouge, and the obtained color of the rouge is applied to the start point. Then, an air brush process is performed, in step S 505 , to make the color density of the rouge applied to the start point thinner (gradate) in proportion to a distance from the start point in order to acquire a color of each pixel at which the rouge covers the skin by adding the thus-obtained density of color of the rouge at each pixel position to the skin color of the pixel at the same position, thereby updating the color of each pixel.  FIG. 67  illustrates a relationship between a distance from the center and a density in the above-mentioned air brush process. 
     Thereafter, in step S 506 , the start point is moved by a predetermined distance in accordance with the movement locus of the start point indicated by the arrows A 1  and A 2 . It should be noted that the movement of the start point from the position P 0  in the direction of the arrow A 1  by a predetermined distance is repeated, and if the moved start point is out of the rouge application area, it returns to the position P 0  and the start point is moved in the direction of the arrow A 2  by the predetermined distance. This distance of movement is, for example, several tens percent of the application size. Moreover, a new application size is calculated in step S 507 . If the makeup pattern is sweet or fresh, the new application size is the same as the previous time, but if the makeup pattern is cool or cute, the new application size decreases at a rate of several percent to several tens percent as the start point moves from the position P 0  in the directions of the arrows A 1  and A 2 . 
     Subsequently, it is determined, in step S 508 , whether or not it is an end point of the rouge application, and if it is not the end point, the above-mentioned steps S 504  to S 507  are repeated, and if it is an end point, the rouge application process is ended. The determination of the end point is such that the rouge application is ended when the start point moves in the direction of the arrow A 2  and out of the rouge application area. 
     As mentioned above, a desired type is selected from a plurality of makeup patterns, an area for applying the rouge is set in the face image in accordance with the selected makeup pattern, and the rouge is applied by overlapping the color of the rouge over the color of the face image in the area where the rouge is applied, and, therefore, the rouge can be applied and displayed in the cheek portion of the face image in accordance with the selected makeup pattern by merely selecting the desired type. 
     Moreover because a gradation is applied by decreasing the density of the rouge as it apart from the start point of the application in the area where the rouge is applied, the applied rouge fits with the skin color of the face image and the eye shadow having no feeling of strangeness can be displayed, and because the start point is sequentially moved in the area where the rouge is applied, the applied rouge fits with the skin color of the face image and the rouge having no feeling of strangeness can be displayed irrespective of a form of the area where the rouge is applied in the face image. 
     The shadow processing part  38  included in the simulator main application  28  may perform a makeup in accordance with a classified and grasped morphological feature of a cheek of a person to be applied with a makeup. Generally, in a cheek makeup, a method of application differs depending on an image to represent. Thus, in the present invention, a consideration was given to only “representation of complexion”. 
     It should be noted that the reason for focusing attention to floridness is based on the following three points. The first reason is that it was considered that an original healthful beauty of each person can be extracted naturally because complexion is an element which anyone possesses in a case of healthy state. 
     Moreover, although there is a clear example of an infant cheek as an example of ruddy cheek, in an example of providing a feeling of “good complexion and preferable”, complexion appears in an area connecting an eye, a nose, a mouth and an ear. Thus, the second reason for focusing attention to complexion is that it was considered a position of applying a makeup can be extracted based on a law common to everyone using an eye, a nose, a mouth and an ear as a guideline. Further, the third reason for focusing attention to complexion is that the representation of complexion aims to have many people using a cheek and is an element of a cheek, which gives a feeling of beauty, and, thus, it meets demands of many people. 
     Moreover, a questionnaire survey was conducted about the purpose of using a cheek for 68 women of 20&#39;s and 30&#39;s who usually apply a makeup. As a result, “in order to show a good complexion” was most selected. Additionally, a result of a questionnaire survey conducted for 40 women of 20&#39;s and 30&#39;s who usually apply a makeup was “suppleness”, “moderately plump” and “good complexion” in an order of descending number of answers. 
     Moreover, when applying a makeup, generally a makeup base, a foundation, etc., are used to repair irregular color in bare skin. In such a case, the complexion naturally present in the bare skin is almost deleted. Accordingly, it is said that adding complexion by a cheek makeup is a natural way of representation to restore the element of originally existing complexion. 
     A cheek makeup, which a cosmetics engineer applies, gives more beautiful finish than that self-applied by an ordinary person. It is considered that this is because a cosmetics engineer learns a method of catching a feature of each face and making a beautiful finish to fits to each person according to a law learned by experience and specialist knowledge. Thus, it was attempted to extract a law by analyzing an application method of a cheek makeup performed by cosmetics engineers. 
     First, a questionnaire survey of “in order to express natural complexion by a cheek makeup, which part is a center of gradation and to which part the gradation is applied?” was carried out for 27 cosmetics engineers by using pictures of faces of 10 models having different cheek features. Additionally, it was asked to indicate the center and range of the portion to which a cheek makeup is applied directly on papers on which the faces are printed. 
       FIG. 68  is an image view for explaining an example of a cheek makeup according to a morphological feature of an object person to be applied with a makeup. In  FIG. 68 , lines to derive the center of a cheek makeup for one model (model A) are illustrated. 
     First, the position of the center of the cheek makeup expressing complexion is near the center of the area, which connects an eye, a nose, a mouth, and an ear. Further, as a result of consideration of extracting the location from elements, which can easily specify a face, as a reference, as illustrated in  FIG. 68 , it was found that by drawing a line  2500  to connect a top of the nose and an auricular (a position in front of an ear, which dents when opening a mouth) and drawing a line  2502  from a point at which a horizontal line  2501  drawing from a center of an eye intersects with a contour of a face, an intersection of both lines coincides with the answers of the cosmetics engineers. 
     Next, a description will be given of a range where a gradation is applied to a cheek makeup. A range where a gradation is applied to a cheek makeup is gradated is within an area connecting an eye, a nose, a mouth and an ear. Further, this range is within a parallelogram, which is drawn using the above-mentioned lines for deriving the center of application of a cheek makeup. 
     Specifically, first, a line  2504  is provided, which is a line (second line) obtained by parallel-moving a line (first line)  2503 , which is drawn from a point at which a horizontal line  2501  passing through the center of the eye when deriving a center of a cheek makeup intersects with a contour of the face, downward to a corner of the mouth. Further, a vertical line (third line)  2505  is drawn upward from the corner of the mouth, and a line (fourth line)  2506  is provided, which is a line parallel-moving the vertical line  2505  to a point at which the horizontal line  2501  passing through the center of the eye intersects with the contour of the face. In  FIG. 68 , the parallelogram is formed by these lines  2503 - 2506 . Additionally, it is appreciated that the rage where the cheek makeup, which a cosmetics engineer applied, is gradated is accommodated in the parallelogram. 
     According to the above-mentioned, it became possible to derive the position of start point to apply a cheek makeup and a target of a gradation range according to an eye, a mouth and a nose, which are tree major elements to determine a space of a face, by using a method common to all faces. It should be noted that, in the following description, the position of a start point to apply a cheek makeup is referred to as “best point”. 
     That is, the start point of the cheek makeup for representation of complexion is the intersection of the line connecting a top of the nose to the aricular and the line drawn to the corner of the mouth from an intersection at which the horizontal line drawn from the center of the eye intersects with the contour of the face. Additionally, the range of gradation is the parallelogram drawn by using the best point as a guide line. Specifically, a line is drawn from the intersection of the horizontal line passing the center of the eye and the contour of the face, and the line is moved downward, in parallel, to the corner of the mouth. Further, a vertical line is drawn upward from the corner of the mouth, and the parallelogram is formed by parallel-moving the vertical line to a point at which the vertical line intersects with the line passing through the center of the eye. 
     As mentioned above, it was found that the best point, which is a start point to apply a cheek makeup and the range of gradation can be derived by a method common to every type. 
     From a cosmetics engineer&#39;s law learned by experience, there was prediction that operating a form of a cheek makeup in consideration of a feature of a cheek makes a fit to that person and gives a beautiful finish. Actually, there may be a case, which gives an ill-fitting finish, even if the derived start point to apply a cheek makeup and the derived range of gradation are satisfied. 
     Thus, in consideration of a direction of adjustment according to a morphological feature of a cheek, an assumption was made to change the cheek makeup according to the directivity thereof. That is, for a person having long cheeks, an adjustment is made to show the cheeks longer, and for a person having short cheeks, an adjustment is made to show the cheeks longer. Additionally, for a person having a remarkable skeleton feeling, an adjustment is made to be seen as plump, and for a person having a remarkable well-fleshed feeling, an adjustment is made to be seen as slimmed-down. 
     Here, a questionnaire survey was conducted in order to verify the above-mentioned assumption. In the questionnaire, two kinds of cheek makeups were given to four models having different morphological features of cheek. One of the cheek makeups is an “OK cheek”, which uses the law of the best point and the target of gradation range and further the form of gradation is changed according to the feature of the cheek. The other of the cheek makeups is an “NG cheek”, which applies the cheek makeup without consideration of the form of gradation. Evaluation of the two kinds was asked to general women (40 persons of 20&#39;s) while not giving information of “OK” and “NG”. 
     Specifically, showing the pictures one by one, and ask an evaluation of the finish of each cheek makeup using a five-point bipolar scale evaluation. Additionally, the evaluation items were “natural-unnatural”, “fit-unfit”, etc. An average value of the evaluations by the 40 persons for the four models was computed and considered. 
     A statistical verification was made about the evaluations, and it was found that the “OK cheek” gives a natural finish and a stereoscopic appearance of the cheek and slightly plump appearance of the cheek flesh more than “NG cheek”. This result is consistent with the above-mentioned element of beautiful cheek, that is, “suppleness and moderately plump”. Additionally, it was found that also the entire balance of the face can be seen as refined by applying the cheek makeup. That is, according to the above-mentioned “OK cheek”, the cheek is seen as stereoscopic, and it is able to feel suppleness of plump skin, and a natural and healthy complexion beauty, which the person originally possesses, can be restored. Additionally, the representation of complexion can adjust a balance while naturally deriving individual beauty. The form of cheek makeup that was applied to each model in the above-mentioned questionnaire is based on the thought of adjusting the appearance of a length and skeleton and flesh of a cheek. 
       FIG. 68  illustrates a form and a range of a cheek makeup according to a morphological feature of the cheek. In  FIG. 68 , the cheek form corresponding to a standard face is an inscribed ellipse  2507 , which is inscribed with a target parallelogram. 
     Here, the inscribed ellipse  2507  of  FIG. 68  has a longer axis A set as a line connecting middle points  2508 - 1  and  2508 - 2  of the vertical sides of the parallelogram, which is a target of gradation, and a shorter axis B set as a line perpendicular to and intersect with at a middle point  2509  of the longer axis A and contacting the upper and lower sides of the parallelogram. Further, if half lengths of the axes A and B are represented as a and b, respectively, the inscribed ellipse is expressed by (x 2 /a 2 )+(y 2 /b 2 )=1. Moreover, in a case of an average face, a relationship is given as a:b=1:0.55±0.02. 
     It should be noted that according to the gradation method of a cheek makeup, basically and actually, an application is performed so that the density at the start point is highest, and is decreased toward the border of the form of gradation, and the gradation in the border portion naturally fits to the skin. 
     (Eyebrow Process) 
     The eyebrow processing part  37  included in the simulator main application  28  can deform an eyebrow to a form, which fits to a face of every user, by performing the deforming process of an eyebrow form of step S 380  in the embodiment 1 based on the following beauty theory. 
     Generally, it is known that a well-shaped eyebrow consists of the following elements. 1) an inner end of eyebrow starts from directly above an inner corner of eye. 2) an outer end of eyebrow is on an extension line connecting a nostril and an inner corner of eye. 3) a peak of eyebrow is at a position ⅔ apart from an inner end of eyebrow. It is said that such an eyebrow is a well-shaped eyebrow. Further, a description will be given in detail, with reference to  FIG. 69 , of these elements. 
       FIG. 69  is an explanatory illustration for explaining a form of an ideal eyebrow. In  FIG. 69 , an inner end of eyebrow  303  exists on an inner corner line of eye  3002 , which rises vertically from an inner corner of eye  3001 , and an outer end of eyebrow exists on a nostril eyebrow outer end line  3006 , which inclines from a nostril  3004  toward an outer corner of eye  3005 , and a peak of eyebrow  301  exists on a vertical eyebrow peak line  3009  at a position apart from the inner end of eyebrow  3003  by about ⅔ to the outer end of eyebrow  3007  along an eyebrow base line  3008 , which connects the inner end of eyebrow  3002  and the outer end of eyebrow  3007 . 
     Although such an eyebrow is a well-shaped eyebrow, it was found that from our recent research that there are many cases in which even if the well-shaped eyebrow is drawn to various users, it does not fits entirely, and other elements are related further. Then, by adding the new elements, it became possible to deform to a beautiful eyebrow, which fits to a feature of each person. 
     The elements are 4) a position of the peak of eyebrow  3010  in the entire face and 5) an angle forming the peak of eyebrow  3010  from the inner end of eyebrow  3003 , and by adding the new elements, an eyebrow can be deformed into a beautiful eyebrow. 
     Here, although the designations of parts such as the inner end of eyebrow  3003 , the peak of eyebrow  3010 , etc., are used, the eyebrow processing part  37  uses the feature point (BR 1 ) corresponding to the inner end of eyebrow  3003  and the feature point (BR 2 ) corresponding to the inner end of eyebrow  3010 . 
     Next, a description will be given of a position of a beautiful eyebrow balanced with a face. For an eyebrow balanced with a face, 4) a position of the peak of eyebrow  3010  in the entire face and 5) an angle forming the peak of eyebrow  3010  from the inner end of eyebrow  303  are important. A description will be given, with reference to  FIG. 70 , of these points.  FIG. 70  is an explanatory illustration for explaining a state of a position of an eyebrow. 
     (A), (B) and(C) of  FIG. 70  illustrate positions occupied by eyebrows in the entire face. Virtual horizontal lines   indicated by solid lines in (A), (B) and (C) of  FIG. 70  are lines indicating a maximum width of the face when viewing the face from a front, and usually they are lines contacting the lower eyelids. Vertical lines are virtual lines, which indicate a maximum width of the face, when the above-mentioned horizontal lines   are vertically extended from a lower portion of the jaw. The horizontal line   and the vertical line   orthogonally cross at the center of the face, and a virtual square is formed by surrounding ends of these horizontal line   and the vertical line   by sides represented by four dotted lines a, b, c and d. 
     Here, paying attention to a position of an eyebrow of (A) of  FIG. 70 , it is appreciated that the peak of eyebrow  3010  is in contact with the side a of the above-mentioned virtual square, and the eyebrow is at a beautiful point when viewing the entire face and well-balanced. On the other hand, in (B) of  FIG. 70 , the peak of eyebrow  3010  exists separate from the side a of the virtual square downward, and the eyes, the nose and the eyebrows are packed into the center portion when viewing the entire face, and the face appears shorter and not balanced. In (C) of  FIG. 70 , the peak of eyebrow  3010  goes beyond the side a of the virtual square, and the face appears longer when viewing the entire face, and an entire balance is not good. 
     Generally, for a person having a round face as a whole, the peak of eyebrow  3010  tends to be positioned under the side a as illustrated in (B) of  FIG. 70 , and for a person having a long face as a whole, the peak of eyebrow  3010  tends to be positioned above the side a as illustrated in (C) of  FIG. 70 , and a case is often seen where the tendencies of face are increased due to a way of drawing of eyebrows. 
     Here, observing the state of (A) of  FIG. 70  in detail, the angle forming the peak of eyebrow  3010  from the inner end of eyebrow  303  is about 10 degrees to the eyebrow base line  3008  connecting the inner end of eyebrow  3003  and the outer end of eyebrow  3007 , and the peak of eyebrow  3010  is in contact with the side a. Additionally, in (B) of  FIG. 70 , the angle forming the peak of eyebrow  3010  is about 5 degrees to the eyebrow base line  3008  connecting the inner end of eyebrow  3003  and the outer end of eyebrow  3007 , and the peak of eyebrow  3010  exists separate from the side a downward. 
     Moreover, in (C) of  FIG. 70 , the angle forming the peak of eyebrow  3010  is about 20 degrees to the eyebrow base line  3008  connecting the inner end of eyebrow  3003  and the outer end of eyebrow  3007 , and the peak of eyebrow  3010  goes beyond the side a. It is appreciated that the eyebrows in the state of (A) or (B) of  FIG. 70  are changed to well-balanced beautiful eyebrows by drawing the eyebrows to be close to the state of (A) of  FIG. 70  as much as possible. 
     A description will be given, with reference to  FIG. 69  and  FIG. 70 , of a procedure of drawing the eyebrow. First, a maximum horizontal width of a front face of a person to be applied is measured, and the virtual horizontal line   having the horizontal width is created along the lower eyelids of the face. Subsequently, a line having the same length as the horizontal line   is extended vertically from a lowermost point of the jaw to create the vertical line  , and the virtual square is formed by orthogonally crossing the horizontal line   and the vertical line   on the front surface of the face to create the virtual sides a, b, c and d passing the ends of each of the horizontal line   and the vertical line  . By comparing the thus-created side a with a position of the peak of eyebrow  3010  of the person to be applied, it is determined whether the peak of eyebrow  3010  coincides with the side a or at a position higher than the side a or at a position lower than the side a. 
     If the peak of eyebrow  3010  coincides with the side a as illustrated in (A) of  FIG. 70 , because the angle of the eyebrow is at an angle of 10 degrees in many cases, a target line is set from the inner end of eyebrow  3003  to the peak of eyebrow  3010 . The target line may be indicated by two points of the inner end of eyebrow  3003  and the peak of eyebrow  3010  by an eyebrow pencil or may be indicated by thin lines. Additionally, the target line may be set under the eyebrow or may be set at the center of the eyebrow or may be set above the eyebrow. In any case, the eyebrow is drawn so that the peak of eyebrow  3010  is on the eyebrow peak line  3009  and the upper edge of the peak of eyebrow  3010  contacts the side a. Although the eyebrow is drawn along the target line with a uniform width up to the peak of eyebrow  3010 , a free curve is created from the peak of eyebrow  3010  to the outer end of eyebrow  3007  in accordance with preference of the person to be applied to complete the eyebrow. 
     Next, as illustrated in (B) of  FIG. 70 , if the peak of eyebrow  3010  is at a position lower than the side a, the eyebrow is drawn so that the peak of eyebrow  3010  is positioned as close as possible to the side a. In this case, because the angle of the line from the inner end of eyebrow  3003  to the peak of eyebrow  3010  in (B) of  FIG. 70  is as small as 5 degrees, in order to shift the peak of eyebrow  3010  to the side a, a target line is set from the inner end of eyebrow  3003  to the peak of eyebrow  3010  by using an eyebrow scale having a drawing part of an angle of 15 degrees. However, if the angle of the target line is excessively large and it shifts greatly from the position of the original peak of eyebrow  3010  of the person to be applied, the person to be applied may feel strangeness in many cases, and, therefore, it is preferable to set a new peak of eyebrow  3010  above the eyebrow peak line  3009  within a range of 2 mm at maximum from the position of the original peak of eyebrow  3010 . If a completely new eyebrow is drawn in a state where an original eyebrow is shaved, there is no need to provide such a limitation. The target line is drawn by an eyebrow pencil, and thereafter, a portion from the inner end of eyebrow  3003  to the peak of eyebrow  3010  is drawn along the target line, and the outer end of eyebrow is drawn to complete an ideal eyebrow. If the original eyebrow is not shaved, the eyebrow is drawn over the original eyebrow as mentioned above, and the eyebrow is trimmed by shaving the stray portion. 
     Moreover, as illustrated in (C) of  FIG. 70 , if the peak of eyebrow  3010  is at a position higher than the side a, the eyebrow is drawn so that the peak of eyebrow  3010  returns to the position on the side a as much as possible. In the case of (C) of  FIG. 70 , because the angle of the line from the inner end of eyebrow  3003  to the peak of eyebrow  3010  is as large as 10 degrees, a target line is set from the inner end of eyebrow  3003  to the peak of eyebrow  3010  with an angle of 8 degrees. Also in such a case, if it shifts greatly from the position of the original peak of eyebrow  3010  of the person to be applied, the person to be applied may feel strangeness in many cases, and, therefore, it is preferable to set a new peak of eyebrow  3010  under the eyebrow peak line  3009  within a range of 2 mm at maximum from the position of the original peak of eyebrow  3010 . If an eyebrow is drawn in the same manner as mentioned above, a beautiful eyebrow well-balanced with the entire face can be created. 
     An image of the face can be adjusted by the drawing method of a portion of eyebrow connecting the inner end of eyebrow  3003  and the peak of eyebrow  3010 . For example, a selection can be done if necessary in response to the demand of the person to be applied, the selection including “look matured” when a thick eyebrow is drawn, “look pretty” if a thin eyebrow is drawn, “look simple” when a straight eyebrow is drawn, and “look gentle” when a curved eyebrow is drawn. 
     As mentioned above, the eyebrow processing part  37  included in the simulator main application  28  can deform an eyebrow into a beautiful eyebrow, which fits to every user, by performing the deforming process of an eyebrow shape of step S 80  in the embodiment 1 based on the above-mentioned beauty theory. 
     (Foundation Process) 
     The foundation processing part  36  included in the simulator main application  28  can perform the following process as a pre-process. 
     A description will be given of a skin color evaluating process procedure as a pre-process of the foundation process.  FIG. 71  is a flowchart illustrating an example of the skin color evaluating process procedure as a pre-process of the foundation process. 
     The skin color evaluating process illustrated in  FIG. 71  inputs an evaluation object image containing a face of which image is taken by an image-taking means such as a camera (S 601 ), and divides the input face image into a predetermined number of portions according to a previously set dividing method (S 602 ). It should be noted that the image obtained by S 601  may be, for example, an image obtained by taking an image of a face, which is entirely illuminated uniformly, by an image-taking apparatus such as a digital camera. 
     Specifically, a lighting box for taking a face image at the same condition is used, and a plurality of halogen lamps are arranged on a front face of the lighting box in order to uniformly illuminate the face within the lighting box, and an image of the face is taken by a TV camera to acquire the taken face image. It should be noted that the image used in the present invention is not limited to a special one, and, for example, an image taken under a normal environment such as under a fluorescent light may be used. 
     Next, a skin color distribution is created from the divided image for each predetermined area (S 603 ), and, for example, a comparison skin color distribution is created using various kinds of data previously accumulated (S 604 ). Additionally, the skin color or the like is compared using the skin color distribution obtained by the process of S 603  and the comparison skin color distribution obtained by the process of S 604  (S 605 ), and an evaluation is performed according to a skin color distribution profile (S 606 ). Additionally, a screen or the like to display to a user is created from the evaluation result obtained by the process of S 606  (S 607 ), and the created screen (contents of evaluation result) is output (S 608 ). 
     Here, it is determined whether to continue the skin color evaluation (S 609 ), and if continue the skin color evaluation (YES in S 609 ), returns to S 602 , and, for example, a division is performed according to a dividing method different from the previous time to perform a process mentioned later. If, in the process of S 609 , the skin color evaluation is not continued (NO in S 609 ), the process is ended. Next, a description will be given of details of the above-mentioned skin color evaluating process. 
     (Face Dividing Process) 
     Next, a description will be given of the above-mentioned face dividing process. The face dividing process performs a predetermined division to the digital image containing the input face.  FIG. 72  is an illustration of an example of the feature points and divided areas. In  FIG. 72 , as an example, the entire face is divided into 90 areas, an average skin color of each area is acquired, a face skin color distribution is represented by the 93 pieces of skin data, and a skin color evaluation is performed according to the distribution, etc. 
     By acquiring many pieces of data for a fixed portion, which is an advantage of a conventional method, for example, a distribution range and an average value of that portion of Japanese women (may be foreign persons (other races), and may be men) can be computed, and, as a result, the skin color data of an individual can be evaluated by comparing with these indexes. Additionally, for example, it becomes possible to perform a skin color comparison before and after a use of cosmetics by the same person and a two-person comparison of skin color with other persons. 
     Here, as an example, the dividing method illustrated in  FIG. 72  has 109 feature points. Additionally, the divided areas illustrated in  FIG. 72  are 93 areas having a triangle shape or a quadrangle shape defined by three or four feature points (for example, indicated by numbers 1-93 in  FIG. 72 ). 
     Moreover,  FIG. 73  is an illustration indicating an example of positional relationship of the 109 feature points in the face corresponding to the above-mentioned  FIG. 72 . Additionally,  FIG. 74  is an illustration of an example of a combination of the feature points constituting each area corresponding to the above-mentioned  FIG. 72 . It should be noted that “No.” and “Name” of each feature point illustrated in  FIG. 73  and “No.” and “structure point” of each area illustrated in  FIG. 74  correspond to the contents illustrated in the above-mentioned  FIG. 72 . 
     Here, when setting the contents of the division, first, the face dividing process sets the feature points of No. 1-37 (for example, indicated by “” in  FIG. 72 ) as first feature points from among the feature points illustrated in  FIG. 73 . It is preferable to arrange the 37 feature points so that 5 points are arranged in the forehead portion, 10 points in the vicinity of left and right eyes, 7 points in the nose, 9 points in the mouth, and 6 points in the face line under the eyes. 
     Next, the face dividing process sets the feature points of No. 38-109 illustrated in  FIG. 73  (for example, points indicated by “Δ” in  FIG. 72 ) as second feature points by using the above-mentioned 37 feature points (first feature points) as a reference. 
     For example, as illustrated in  FIG. 73 , there are a total of 109 points including the feature points  1 - 37  of No. 1-37 previously defined, the points  38 - 49  acquired by intersections of a plurality of lines passing between at least two feature points from among the feature points, the points  50 - 57  and  67 - 109  internally dividing the lines between two points, and the points  58 - 66  on the lines passing two feature points and having the same vertical coordinate or horizontal coordinate as a certain specific point. 
     Moreover, it is divided into areas each encircled by at least three points as structure points as illustrated in  FIG. 74  from among the first and second feature points (109 points). It should be noted that the number of points defining each area may be three points or four points as illustrated in  FIG. 74  or may be five points. 
     Here, each area (area No. 1-93) illustrated in  FIG. 72  is set so that a division having a physiological sense can be done based on the experience of observing many skin colors. That is, by setting as illustrated in  FIG. 72 , a division is done so that the area is narrow in a portion where color heterogeneity tends to occur, and the area is wide in other portions. 
     Specifically, in the example division illustrated in  FIGS. 72-74 , for example, the divided area is set to be wide in the portion of the forehead, and the area is set to be narrow in the vicinity of the eyes and mouth and the cheeks. The thus-set divided areas permit an evaluation in more detail and with a higher accuracy by setting the area, which is an important portion (area) in evaluating skin color, to be narrow. 
     (Division of Face and Creation of Face Skin Color Distribution) 
     Next, a description will be given specifically of creation of a face skin color distribution with respect to the divided face. It should be noted that although an image-taking apparatus using a lighting apparatus, which uniformly illuminates an entire face, and a digital camera is explained with an example of creating a skin color distribution with respect to a face image acquired by using, for example, “Masuda et al., Development of Quantization System of Spots and Freckles Using Image Analysis, Shougishi, V28, N2, 1994”, etc., the image-taking method of an image-taken digital image used in the present invention is not specially limited to this. 
     A total of 109 feature points can be computed by taking a picture of a person to be examined set at a predetermine position and designating, for example, 37 first feature points according to the face dividing process as mentioned above. Additionally, the face dividing process divides it into 93 areas from the 109 feature points by the above-mentioned setting as illustrated in  FIG. 74 . It should be noted that the skin color distribution is created by an average value for each area by using at least one of L*, a*, b*, Cab*, and hab in a L*a*b* display system, each value of three stimulation values X, Y, and Z, each value of RGB, color phase H, brightness V, color saturation C, amount of melanin, and amount of hemoglobin. 
     Because a portion which is not in a skin color is excluded from the object to be evaluated in outside the entire area and even in the area, such a portion is colored by a specific color such as, for example, a cyan color or the like, which separates away greatly from a skin color. Further, information of details of skin is deleted so that the face skin color distribution is easily grasped, and, for example, it can be appreciated that the model A has a feature that “the skin color around the eye is dark”. It should be noted that because a peripheral portion of the image-taken face may have low uniformity of illumination, data of the peripheral portion may be excluded in the skin color evaluating process. 
     Additionally, if an “average face” is divided into areas by the same method, and each area is colored by obtained  93  skin colors of the model A, pure color information excluding face form information of the model can be grasped. 
     Because the evaluation can be done according to the divided areas as a reference by performing the above-mentioned process, face form information of a person can be excluded, and a comparison in a skin color distribution between persons having different face forms can be performed easily. Thus, using this feature, for example, an evaluation can be done by comparing the face skin color distribution of the model A with an average value of the same generation. 
     (Creation of Face Skin Color Distribution for Comparison) 
     Next, a description will be given of an example of creating a face skin color distribution for comparison in the skin color distribution evaluating process. As an average skin color distribution for each generation, a face image of a person of a corresponding generation are subject to an area fragmentation, and, thereafter, a skin color distribution is acquired according to average values by using at least one of L*, a*, b*, Cab*, and hab in a L*a*b* display system, each value of three stimulation values X, Y, and Z, each value of RGB, color phase H, brightness V, color saturation C, amount of melanin, and amount of hemoglobin, and an average skin color distribution is acquired by computing an average value for each generation. 
     It should be noted that each area of the average face is colored by using acquired data of the average skin color distribution of each generation of Japanese women from 20&#39;s to 60&#39;s. By displaying it with coloring, an average skin color distribution for each generation can be created, which permits a highly accurate skin color evaluation of the evaluation object image by comparing with the data. 
     (Comparison of Skin Color Distribution) 
     Next, a description will be given specifically of a comparison example of a skin color distribution. The skin color distribution evaluating process performs a comparison of a skin color distribution by taking differences. A face skin color distribution can be grasped by dividing the face image into predetermined areas, and an easily recognizable display, which includes only color information by excluding a form of a face by replacing the color with a standard face such as an average face. Because it becomes possible to acquire a difference value between two persons, such as average value data of persons belonging to a certain category (classified by age, occupation, or gender), data of an ideal person such as a talent, data of other persons, it can be used for counseling at a time of sales of cosmetics. 
     (Aggregation (Grouping) of Skin Color Distribution and Example of Creating Skin Color Distribution Profile) 
     Here, in the present embodiment, the skin color distribution evaluating process can aggregate (group) areas having similar color tendencies in the skin color areas by acquiring a main component by applying a main component analysis to previous data or the like. Thereby, an evaluation can be done easily for each group.  FIG. 75  is an illustration illustrating an example of area numbers constituting each group. It should be noted that the feature points constituting the areas illustrated in FIG.  75  correspond to the above-mentioned  FIGS. 72-74 . 
     In the example illustrated in  FIG. 75 , the grouping is applied to a portion (1) under cheek, (2) cheek front, (3) eyelid and dark ring portion under eye, (4) forehead, (5) round about nose, and (6) round about mouth. Additionally, as a feature of color, “(1) under cheek” has high-brightness, “(2) cheek front” is on the side of red and has slightly high-brightness, “(3) eyelid and dark ring portion under eye” is on the side of yellow and has a slightly low-brightness, “(4) forehead” is on a side of yellow and has slightly high-brightness, “(5) round about nose” is on the side of red and has slightly-low brightness, and “(6) round about mouth” is on the side of red and has low-brightness. 
     Here, as a result of main component analysis of color phase H performed on the effective  57 , in which 4 areas of lips are excluded, for 59 persons of age 20-67 as an example, it was found that 90.1% of 57 pieces of data are explainable with 6 main components. 
     Thus, the 57 areas are classified according to the above-mentioned main components into the above-mentioned (1) under cheek, (2) cheek front, (3) eyelid and dark ring portion under eye, (4) forehead, (5) round about nose, and (6) round about mouth. Additionally, an evaluation of a skin color distribution can be done according to a balance of the main component score (skin color profile). 
     Next, a description will be given of a face classification process procedure as a pre-process of the foundation process. Here, a consideration was given on what stereoscopic feel is evaluated as beautiful in order to extract a method of representing a beauty by adjusting stereoscopic feeling. When the consideration is given, faces of (a) natural face, (b) natural base make naturally trimming skin color heterogeneity, and (c) egg type base make adjusting stereoscopic feeling so show an entire face with an average face balance were used. 
     Here, the three types of base make of the above-mentioned (a)-(c) were applied to 6 models having different features of face, and a questionnaire survey was conducted for 20 women of 20&#39;s using face pictures taken from the faces. 
     An evaluation was performed according to 8 items concerning appearance of face such as (appearance of forehead, stereoscopic feeling of forehead, degree of level of bridge of nose, flesh of cheek, length of cheek, face line, degree of protrusion of jaw, balance of eyes and nose) and 3 items concerning a collective impression such as (stereoscopic feeling, beautifulness, and favorability of an entire face). Additionally, an answer was asked as to whether appearance of face is appropriate. 
     (Consideration According to Standard Balanced Face) 
     As a condition of general beauty face, there are an egg-type face line and a golden balance. In the golden balance, for example, a position of eyes is at about ½ of a height of an entire head, and an inner end of eyebrow is positioned at ⅓ from a hairline within a range from the hairline to a tip of a jaw, and a nostril is positioned at ⅔ from the hairline. 
     Here, it is known that the balance of the “average face”, which is created by averaging size information and color information of face pictures of a plurality of persons according to a conventional image synthesizing technique becomes close to the value of the golden balance. Additionally, it is disclosed that if data corresponding to 10 persons is used to create an average face, there is less change in impression from one created from face pictures of different 10 persons (For example, Nishiya Kazumi et al., “Research Features of Average Face”, Japanese psychology Society No. 63 conference papers, issued August, 1999.) The “average face” created using face pictures of 40 women satisfies the above-mentioned golden balance. In the following, the golden balance is set as a standard balance. 
     Moreover, referring to a face picture in which information regarding depth and skeleton flesh feeling (and also skin color information) is deleted from the “average face”, it is appreciated that the depth and skeleton flesh feeling gives a great influence to an impression of a face. Additionally, it was found that an egg form similar to the face line can be extracted from inside the face line when the above-mentioned image analysis (monochrome posterization process) was applied to the average face. 
     Here,  FIG. 76  is a view illustrating an example of a result of an image (or stereoscopic feeling) analysis of the average face. As illustrated in  FIG. 76 , it was found that a face line (inner face line)  4001 , which is obtained by analogously reducing an outer face line  4000 , exists inside the outer face line  4000  corresponding to the face line of the face. Accordingly, it is indicated that showing the stereoscopic feeling of an inner side of a face to be an egg form is one of representing methods of new beauty. 
     (Adjustment of Stereoscopic Feeling—Best Oval Adjusting Method) 
     Next, in order to create a cosmetology to create stereoscopic feeling inside the face line, that is, trimming the inner face line to be an egg form, an adjustment hypothesis was set up. Here, the forms of the outer face line and the inner face line extracted from an average face are defined as “best oval”. The outer face line is an egg-type form (standard outer face line) having a ratio of horizontal width to vertical width of about 1:1.4. The inner face line of the “average face” is similar to the standard outer face line, and is a form reduced by a predetermined ratio, and a ratio of horizontal width to vertical width is about 1:1.4. 
     The outer face line  4000  is a form indicated by a relationship of “ratio of horizontal width of face: vertical width of face=1:1.4” as mentioned above. It should be noted that the outer line face balance is a point for determining a directivity of space adjustment of an entire face. Additionally, in a case of applying the inner face line  4001  to an individual face, as illustrated in  FIG. 76 , first, a horizontal width between the left and right cheek bones of an individual face is set to 1. Then, the standard inner face line  4001  having a vertical width of about 1.4, that is, the best oval, is drawn using the horizontal width as a reference, and positions to be in contact with outer ends of eyebrow of each face (an extension line passing from a side of a nostril to en outer end of eyebrow as a target). It should be noted that the inner face line balance is an adjustment zone for stereoscopic feeling and a space due to skeleton and flesh. 
     Additionally, a space between the outer face line  4000  and the inner face line  4001  is set as a zone (hutched part in  FIG. 76 )  4002  for providing a natural depth. Additionally, the zone  4002  is an adjustment zone for adjusting a depth to provide a depth and simultaneously show the forms of the outer face line  4000  and the inner face line  4001  well-shaped. 
     The above-mentioned pre-process of the foundation process may be used for a base makeup. 
     As mentioned above, according to the present invention, a makeup can be applied correctly to a face of a user contained in a dynamic image at a small processing load. It should be noted that the picture-taking means recited in the claims corresponds to the camera  2 , the control means corresponds to the control part  8 , the display means corresponds to the monitor  7 , the face recognition processing means corresponds to the face recognition processing part  33 , the makeup processing means corresponds to the makeup processing part  35 , the operation means corresponds to the operation panel  4 , the half mirror means corresponds to the half mirror  3 , and the print means corresponds to the printer  5 . 
     Moreover, the present invention is not limited to the specifically disclosed embodiments, and variations and modifications may be made without departing from the scope of the claims. 
     Because the makeup simulation apparatus  1  according to the present invention performs a realtime simulation, the makeup simulation apparatus  1  can recognize instantly the tracking points of the face and a simulation is performed based on the tracking points, which is different from the one using a static image or a conventional makeup simulation apparatus, and, therefore, the makeup simulation apparatus  1  permits the followings. 
     The makeup simulation apparatus  1  according to the present invention permits a realtime simulation. The makeup simulation apparatus  1  according to the present invention is capable of performing a simulation of not only a front face as in a conventional one but also a side face, and, thus, it is easy to check a simulation effect and a technique of rouge or the like. 
     Because the makeup simulation apparatus  1  according to the present invention can perform a realtime simulation, it becomes possible to perform an analysis to recognize a face as a solid, unlike a conventional planar expression, and representation of stereoscopic feeling and texture. 
     Moreover, because the makeup simulation apparatus  1  according to the present invention can perform face recognition for many persons at the same time, a realtime simulation for many persons can be performed at the same time. Because the makeup simulation apparatus  1  according to the present invention is excellent in the face recognizing function, a makeup can be applied according to features and classification by conforming to a feature of an individual or by automatically classifying men and women. For example, the makeup simulation apparatus  1  according to the present invention can perform a makeup simulation for each of a couple at the same time. 
     The present international application claims priority based on Japanese patent application No. 2007-208809 filed Aug. 10, 2007 and Japanese patent application No. 2008-202449 filed Aug. 5, 2008, the entire contents of No. 2007-208809 and No. 2008-202449 are incorporated in the present international application by reference.