SERVER AND METHOD OF PROVIDING DATA

A server communicatively connected with a customer terminal used by a customer and an operator terminal directing the customer includes a skin image data database that associates and stores skin image data on the customer, cosmetic data on the type of a cosmetic, and made-up skin image data on the customer indicated from the skin image data who has used the cosmetic indicated from the cosmetic data; receives skin image data imaged with the camera of the customer terminal; receives cosmetic data selected by the operator from the operator terminal; and searches the skin image data database based on the received skin image data and cosmetic data and extracts made-up skin image data corresponding to the received cosmetic data.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priority to Japanese Patent Application No. 2015-147280 filed on Jul. 25, 2015, the entire contents of which are incorporated by reference herein.

TECHNICAL FIELD

The present invention relates to a server communicatively connected with a customer terminal used by a customer and an operator terminal directing the customer, and a method of providing data.

BACKGROUND ART

Recently, the simulators displaying the image of a subject's virtually made-up face on the screen have been known for the research and development of cosmetics or the make-up advice in shops.

Such simulators take a subject's face image and superimpose the image of a made-up face on the face image (refer to Patent Document 1).

CITATION LIST

Patent Literature

Patent Document 1: JP 2009-53981 A

SUMMARY OF INVENTION

Patent Document 1 extracts the feature point of a face image, compares the feature point of a two-dimensionally projected standard made-up form with this extracted feature point, distorts the standard made-up form so that the feature point of the two-dimensionally projected made-up form corresponds to the subject's feature point, superimposes the distorted standard made-up form on the subject's face image, so as to generate the image of a virtually made-up face that the subject desires.

However, such a constitution may increase the processing time because it requires complex processes such as the extraction of the feature point, and the formation of and the distortion of the standard made-up form.

In view of the above-mentioned problems, an objective of the present invention is to provide a server and a method of providing data that are capable of extracting a customer's skin image after the makeup in a short time.

The first aspect of the present invention provides a server communicatively connected with a customer terminal used by a customer and an operator terminal directing the customer, including:

a skin image data database that associates and stores skin image data on the customer, cosmetic data on the type of a cosmetic, and made-up skin image data on the customer indicated from the skin image data who has used the cosmetic indicated from the cosmetic data;

a skin image data receiving unit that receives skin image data imaged with the camera of the customer terminal;

a cosmetic data receiving unit that receives cosmetic data selected by the operator from the operator terminal; and

a made-up skin image extracting unit that searches the skin image data database based on the received skin image data and cosmetic data and extracts made-up skin image data corresponding to the received cosmetic data.

According to the first aspect of the present invention, a server communicatively connected with a customer terminal used by a customer and an operator terminal directing the customer includes a skin image data database that associates and stores skin image data on the customer, cosmetic data on the type of a cosmetic, and made-up skin image data on the customer indicated from the skin image data who has used the cosmetic indicated from the cosmetic data; receives skin image data imaged with the camera of the customer terminal; receives cosmetic data selected by the operator from the operator terminal; and searches the skin image data database based on the received skin image data and cosmetic data and extracts made-up skin image data corresponding to the received cosmetic data.

The first aspect of the present invention falls into the category of a server, but the category of a method of providing data has the same functions and effects.

The second aspect of the present invention provides the server according to the first aspect of the present invention further including: a face image data receiving unit that receives face image data on the customer's face from the customer terminal, the face image data being imaged with a camera; a data synthesizing unit that synthesizes the face image data by superimposing the extracted made-up skin image data on the face image data; and a face image data transmitting unit that transmits the synthesized face image data to the customer terminal.

According to the second aspect of the present invention, the server according to the first aspect of the present invention receives face image data on the customer's face from the customer terminal, the face image data being imaged with a camera; synthesizes the face image data by superimposing the extracted made-up skin image data on the face image data; and transmits the synthesized face image data to the customer terminal.

The third aspect of the present invention provides the server according to the first aspect of the present invention, further including: a partial face image data receiving unit that receives partial face image data on the customer's partial face from the customer terminal, the partial face image data being imaged with a camera; a data synthesizing unit that synthesizes the partial face image data by superimposing the extracted made-up skin image data on the partial face image data; and a partial face image data transmitting unit that transmits the synthesized partial face image data to the customer terminal.

According to the third aspect of the present invention, the server according to the first aspect of the present invention receives partial face image data on the customer's partial face from the customer terminal, the partial face image data being imaged with a camera; synthesizes the partial face image data by superimposing the extracted made-up skin image data on the partial face image data; and transmits the synthesized partial face image data to the customer terminal.

The fourth aspect of the present invention provides a method of providing data executed by a server communicatively connected with a customer terminal used by a customer and an operator terminal directing the customer, the server being provided with

a skin image data database that associates and stores skin image data on the customer, cosmetic data on the type of a cosmetic, and made-up skin image data on the customer indicated from the skin image data who has used the cosmetic indicated from the cosmetic data, the method including the steps of

receiving skin image data imaged with the camera of the customer terminal;

receiving cosmetic data selected by the operator from the operator terminal; and

searching the skin image data database based on the received skin image data and cosmetic data and extracting made-up skin image data corresponding to the received cosmetic data.

The present invention can provide a server and a method of providing data that are capable of extracting a customer's skin image after the makeup in a short time.

DESCRIPTION OF EMBODIMENTS

Embodiments of the present invention will be described below with reference to the attached drawings. However, this is illustrative only, and the technological scope of the present invention is not limited thereto.

Configuration of Data Providing System1

FIG. 1shows an overview of the data providing system1according to a preferable embodiment of the present invention. The data providing system1includes a server10, a customer terminal100, and an operator terminal200.

The server10is communicatively connected with a customer terminal100used by a customer and an operator terminal200directing the customer. The server10has a skin image data database that associates and stores skin image data on the customer, cosmetic data on the type of a cosmetic, and made-up skin image data on the customer indicated from the skin image data who has used the cosmetic indicated from the cosmetic data. The server10receives skin image data imaged with the camera of the customer terminal100. The server10receives cosmetic data selected by the operator from the operator terminal200. The server10searches the skin image data database based on the received skin image data and cosmetic data and extracts made-up skin image data corresponding to the received cosmetic data.

The server10also receives face image data on the customer's face imaged with the camera of the customer terminal100. The server10synthesizes the face image data by superimposing the extracted made-up skin image data on the face image data. The server10transmits the synthesized face image data to the customer terminal100.

The server10also receives partial face image data on the customer's partial face imaged with the camera of the customer terminal100. The server10synthesizes the partial face image data by superimposing the extracted made-up skin image data on the partial face image data. The server10transmits the synthesized partial face image data to the customer terminal100.

The customer terminal100and the operator terminal200are communicatively connected with the server10.

The server10generates a skin image data database that associates unmade-up skin image data on the unmade-up skin image of the customer, cosmetic data on the type of a cosmetic, and made-up skin image data on the customer indicated from the skin image data who has used the cosmetic indicated from the cosmetic data (step S01). The server10acquires unmade-up skin image data, cosmetic data, and made-up skin image data through a public line network5such as the Internet and generates the skin image data database. The server10may acquire image data on the customer's face imaged with the imaging device that the customer terminal100or another terminal has as the unmade-up skin image data, cosmetic data on the cosmetic that this customer has actually used, and image data on the customer's face after the makeup as made-up skin image data to associate these data with each other and generate the skin image data database.

The customer terminal100images the customer's face with the camera and transmits the image of the face to the server10as face image data (step S02). The camera of the customer terminal100images the customer's entire or partial face such as the cheek, forehead, mouth, or nose.

The server10recognizes the received face image data and extracts skin image data. If the received face image data are on an entire face, the server10extracts the respective feature points of the parts to acquire skin image data on the parts. If the received face image data are on a partial face, the server10extracts the feature point of this part to acquire skin image data on the part.

The server10searches the skin image data database to extract unmade-up skin image data similar to the extracted skin image data and transmits the unmade-up skin image data to the operator terminal200(step S03). In the step S03, the server10recognizes the received skin image data, compares a plurality of feature points existing in the skin image data with a plurality of feature points existing in the unmade-up skin image data stored in the skin image data database, and extracts unmade-up skin image data similar to the skin image data. The server10may extract unmade-up skin image data similar to the received skin image data by another method.

The operator terminal200displays a skin image based on the received unmade-up skin image data, selects a cosmetic to be applied to this skin image, and transmits cosmetic data on the selected cosmetic to the server10(step S04).

The server10receives cosmetic data transmitted from the operator terminal200. The server10searches the skin image data database based on the skin image data received from the customer terminal100and the cosmetic data received from the operator terminal200and extracts made-up skin image data corresponding to the received cosmetic data.

The server10synthesizes the face image data by superimposing the extracted made-up skin image data on the face image data received from the customer terminal100to generate synthesized face image data. If the face image data is on an entire face, the server10superimposes the made-up skin image data on the corresponding parts or the application parts indicated from the cosmetic data. If the face image data is on a partial face, the server10superimposes the made-up skin image data on this part.

The server10transmits the synthesized face image data to the customer terminal100(Step S05).

The customer terminal100receives and displays the synthesized face image data transmitted from the server10.

Configuration of Data Providing System1

FIG. 2shows a system configuration diagram of the data providing system1according to a preferable embodiment of the present invention. The data providing system1includes a server10, a customer terminal100, an operator server200, and a public line network5(e.g. the Internet network, a third and a fourth generation networks).

The server10is a server device capable of data communication with the customer terminal100and the operator terminal200. The server10has a skin image data database that associates and stores skin image data on the unmade-up skin image of the customer, cosmetic data on the type of a cosmetic, and made-up skin image data on the skin image data after the cosmetic indicated from the cosmetic data has been used. The server10receives skin image data imaged with the camera of the customer terminal100. The server10receives cosmetic data selected by the operator from the operator terminal200. The server10searches the skin image data database based on the received skin image data and cosmetic data and extracts made-up skin image data corresponding to the received cosmetic data.

The server10also receives face image data on the customer's face imaged with the camera of the customer terminal100. The server10synthesizes the face image data by superimposing the extracted made-up skin image data on the face image data. The server10transmits the synthesized face image data to the customer terminal100.

The server10also receives partial face image data on the customer's partial face imaged with the camera of the customer terminal100. The server10synthesizes the partial face image data by superimposing the extracted made-up skin image data on the partial face image data. The server10transmits the synthesized partial face image data to the customer terminal100.

The customer terminal100has functions to be described later and a capability of data communication, which is a home or office appliance that is carried and moved. Examples of the customer terminal100may include information appliances such as a mobile phone, a mobile information terminal, a smart phone, a notebook computer, a tablet terminal, a net book terminal, a slate terminal, an electronic book terminal, and a portable music player. The customer terminal100may be a fixed terminal installed in stores, facilities, etc.

The operator terminal200has functions to be described later and a capability of data communication, which is a home or office appliance that is carried and moved or a stationary appliance. Examples of the operator terminal200may include information appliances such as a mobile phone, a mobile information terminal, a smart phone, a notebook computer, a tablet terminal, a net book terminal, a slate terminal, an electronic book terminal, a portable music player, and a desktop. The operator terminal200may be a fixed terminal installed in stores, facilities, etc.

Functions

FIG. 3shows a functional block diagram of the server10, the customer terminal100, and the operator terminal200to illustrate the relationship among their respective functions.

The server10includes a control unit11including a central processing unit (hereinafter referred to as “CPU”), a random access memory (hereinafter referred to as “RAM”), and a read only memory (hereinafter referred to as “ROM”); and a communication unit12including a Wireless Fidelity or Wi-Fi® enabled device complying with, for example, IEEE 802.11, or a wireless device complying with the IMT-2000 standard such as the third generation mobile communication system. The communication unit12may include a wired device for LAN connection.

The server10also includes a memory unit13such as a hard disk, a semiconductor memory, a record medium, or a memory card to store data. The memory unit13includes a skin image data database to be described later.

The server10also includes a device recognizing image data as a synthesis unit14or a device combining data on a plurality of images to synthesize data on one image.

In the server10, the control unit11reads a predetermined program to run a database generation module20, a face image data receiving module21, a skin image data transmitting module22, a cosmetic data receiving module23, a synthesized data transmitting module24, and an operator image transceiving module25in cooperation with the communication unit12. Furthermore, in the server10, the control unit11reads a predetermined program to run a database storing module30in cooperation with the memory unit13. Still furthermore, in the server10, the control unit11reads a predetermined program to run a data extracting module40and a data synthesis module41in cooperation with the synthesis unit14.

The customer terminal100includes a control unit110including a CPU, a RAM, and a ROM; and a communication unit120such as a device capable of communicating with other devices, for example, a Wireless Fidelity or Wi-Fi® enabled device complying with IEEE 802.11 in the same way as the server10. The customer terminal100also includes a device capable of wired connection.

The customer terminal100also includes an input-output unit130including a display unit outputting and displaying data and images that have been processed by the control unit110; and an input unit such as a touch panel, a keyboard, or a mouse that receive an input from the customer. The customer terminal100also includes a device such as a camera to image the entire or partial face of the customer. The customer terminal100also includes a device capable of acquiring location information, such as a GPS.

In the customer terminal100, the control unit110reads a predetermined program to run a face image data transmitting module150, a synthesized data receiving module151, and an operator image data receiving module152in cooperation with the communication unit120. Furthermore, in the customer terminal100, the control unit110reads a predetermined program to run an imaging module160and a display module161in cooperation with the input-output unit130.

The operator terminal200includes a control unit210including a CPU, a RAM, and a ROM; and a communication unit220such as a device capable of communicating with other devices, for example, a Wi-Fi® enabled device complying with IEEE 802.11 in the same way as the customer terminal100. The operator terminal200also includes a device capable of wired connection.

The operator terminal200also includes an input-output unit230including a display unit outputting and displaying data and images that have been processed by the control unit210; and an input unit such as a touch panel, a keyboard, or a mouse that receive an input from the operator. The operator terminal200also includes a device such as a camera to image the operator's face. The operator terminal200also includes a device capable of acquiring location information, such as a GPS.

In the operator terminal200, the control unit210reads a predetermined program to run a skin image data receiving module250, a cosmetic data transmitting module251, and an operator image data transmitting module252in cooperation with the communication unit220. Furthermore, in the operator terminal200, the control unit210reads a predetermined program to run a display module260and an imaging module261in cooperation with the input-output unit230

Data Providing Process

FIG. 4shows a flow chart of the data providing process executed by the server10, the customer terminal100, and the operator terminal200. The tasks executed by the modules of each of the above-mentioned units will be explained below together with this process.

First, the database generation module20of the server10generates a skin image data database that associates unmade-up skin image data on the unmade-up skin image of the customer, cosmetic data on the type of a cosmetic, and made-up skin image data on the skin image of the customer indicated from the unmade-up skin image data who has used the cosmetic indicated from the cosmetic data (step S10). The database generation module20acquires and associates a plurality of unmade-up skin image data, cosmetic data, and made-up skin image data to generate a skin image data database.

In the step S10, the database generation module20of the server10acquires unmade-up skin image data, cosmetic data, and made-up skin image data through a public line network5such as the Internet and generates the skin image data database. Moreover, the unmade-up skin image data that the database generation module20acquires may be on an entire or a partial face such as the eye, mouth, cheek, forehead, or nose. The database generation module20may acquire image data on the customer's face imaged with the imaging device such as a camera that the customer terminal100or the operator terminal200has as the unmade-up skin image data, cosmetic data on the cosmetic that this customer has actually used, and image data on the customer's face after the makeup as made-up skin image data to associate these data with each other and generate the skin image data database.

The database storing module30of the server10stores the skin image data database generated by the database generation module20as a skin image data database as shown inFIG. 5(step S11).

Skin Image Data Database

FIG. 5shows the skin image data database that the database storing module30of the server10stores. The database storing module30associates and stores the above-mentioned unmade-up skin image data, cosmetic data, and made-up skin image data. InFIG. 5, the database storing module30associates and stores the skin image A as the unmade-up skin image data, the cosmetic data1111as cosmetic data, and the made-up skin image a as the made-up skin image data. The database storing module30also associates and stores the skin image B as the unmade-up skin image data, the cosmetic data1112as cosmetic data, and the made-up skin image b as the made-up skin image data. The database storing module30also associates and stores the skin image C as the unmade-up skin image data, the cosmetic data1113as cosmetic data, and the made-up skin image c as the made-up skin image data. Furthermore, the database storing module30associates and stores other unmade-up skin image data, cosmetic data, and made-up skin image data. The unmade-up skin image data stored by the database storing module30indicate the skin image of the entire or partial face of a customer. The cosmetic data stored by the database storing module30indicate a cosmetic to be used. Examples of the cosmetic data stored by the database storing module30include a foundation, an eyebrow pencil, a blusher, a lipstick, a gross, an eyeliner, an eye shadow, and an eyelash liner. The made-up skin image data stored by the database storing module30indicates an entire or partial face to which the cosmetic indicated from the cosmetic data has been applied.

First, the imaging module160of the customer terminal100judges whether or not the imaging module160has received an input of operation to image the customer's face (step S12). In the step S12, the imaging module160images the entire or partial face of the customer. Examples of the partial face of the customer that is imaged by the imaging module160include the eye, nose, mouth, cheek, and forehead. In the step S12, if judging that the imaging module160has not received an input of the operation (NO), the imaging module160repeats the process of this step until receiving an input of the operation.

On the other hand, if judging that the imaging module160has received an input of operation to image the customer's face (YES) in the step S12, the imaging module160of the customer terminal100images the customer's face (step S13).

The display module161of the customer terminal100displays the face image of the customer, a transmission icon, and a modification icon as shown inFIGS. 6 to 9(step S14).

The face image of the customer imaged by the imaging module160of the customer terminal100and displayed by the display module161of the customer terminal100will be explained below with reference toFIGS. 6 to 9.

FIG. 6shows the case where the imaging module160of the customer terminal100images the entire face of the customer. In this case, the display module161displays the imaged entire face of the customer as a face image410. The display module161displays a modification icon440and a transmission icon400. If the display module161has received an input of operation to the transmission icon400, the face image data transmitting module150of the customer terminal100transmits the face image data on the face image to the server10. Furthermore, if the display module161has received an input of operation to the modification icon440, the imaging module160images the customer's face again.

FIG. 7shows the case where the imaging module160of the customer terminal100images the partial face of the customer. In this embodiment, the imaging module160images the forehead of the customer. In this case, the display module161displays the imaged partial face of the customer as a partial face image411. The display module161displays a modification icon440and a transmission icon400. If the display module161has received an input of operation to the transmission icon400, the face image data transmitting module150of the customer terminal100transmits the partial face image data on the partial face image to the server10. Furthermore, if the display module161has received an input of operation to the modification icon440, the imaging module160images the customer's face again.

FIG. 8shows the case where the imaging module160of the customer terminal100images the partial face of the customer. In this embodiment, the imaging module160images the cheek and the nose of the customer. In this case, the display module161displays the imaged partial face of the customer as a partial face image412. The display module161displays a modification icon440and a transmission icon400. If the display module161has received an input of operation to the transmission icon400, the face image data transmitting module150of the customer terminal100transmits the partial face image data on the partial face image to the server10. Furthermore, if the display module161has received an input of operation to the modification icon440, the imaging module160images the customer's face again.

FIG. 9shows the case where the imaging module160of the customer terminal100images the partial face of the customer. In this embodiment, the imaging module160images the mouth of the customer. In this case, the display module161displays the imaged partial face of the customer as a partial face image413. The display module161displays a modification icon440and a transmission icon400. If the display module161has received an input of operation to the transmission icon400, the face image data transmitting module150of the customer terminal100transmits the partial face image data on the partial face image to the server10. Furthermore, if the display module161has received an input of operation to the modification icon440, the imaging module160images the customer's face again.

The display module161of the customer terminal100judges whether or not the display module161has received an input of operation to the modification icon440(step S15). In the step S15, if judging that the display module161has received an input of operation to the modification icon440(YES), the display module161images the customer's face again.

On the other hand, if judging that the display module161has not received an input of operation to the modification icon440(YES) in the step S15, the display module161of the customer terminal100judges whether or not the display module161has received an input of operation to the transmission icon400(step S16). If judging that the display module161has not received an input of operation to the transmission icon400(NO) in the step S16, the display module161repeats the process of the above-mentioned step S15.

If the display module161judges that the display module161has received an input of operation to the transmission icon400(YES) in the step S16, the face image data transmitting module150of the customer terminal100transmits the face image data imaged by the imaging module160to the server10(step S17).

The face image data receiving module21of the server10receives the face image data transmitted from the customer terminal100. The data extracting module40of the server10recognizes the received face image data and extracts skin image data from the face image data (step S18). If the face image data that the face image data receiving module21has received is the entire face of a customer, the data extracting module40extracts feature points from the received face image data, specifies the specific parts such as the eyes, mouth, nose, cheek, and forehead in the face image data, and extracts skin image data on the parts. On the other hand, if the face image data that the face image data receiving module21has received is the partial face of a customer, the data extracting module40extracts a feature point from the received face image data, specifies the part indicated from the face image data, and extracts skin image data on this part.

The data extracting module40of the server10searches the skin image data database to extract unmade-up skin image data similar to the skin image data from the received face image data (step S19). In the step S19, the data extracting module40recognizes the skin image data extracted in the step S18, and retrieves and extracts unmade-up skin image data with a similar feature point to that of the skin image data from the skin image data database. The data extracting module40may extract unmade-up skin image data similar to the skin image data by another method.

The skin image data transmitting module22of the server10transmits the unmade-up skin image data extracted from the skin image data database by the data extracting module40to the operator terminal200(step S20).

The skin image data receiving module250of the operator terminal200receives the skin image data transmitted from the server10. The display module260of the operator terminal200displays a skin image420and a cosmetic data window450based on the received skin image data as shown inFIG. 10(step S21). In the step S21, the display module260displays cosmetic data1111to1120indicating cosmetics to be applied to the skin image420in the cosmetic data window450. The cosmetic data1111to1120that the display module260displays are stored in the skin image data database. The number of cosmetic data that the display module260displays is not limited to the number of this embodiment and may be more or less than this number. In the step S21, the display module260may display the skin image of not an entire but a partial face. In this case, only the skin image of the corresponding part is displayed.

The display module260of the operator terminal200judges whether or not the display module260has received an input from the operator to select cosmetic data (step S22). In the step S22, if judging that the display module260has not received an input from the operator to select cosmetic data (NO), the display module260repeats the process of this step until receiving an input of the selection.

On the other hand, if judging that the display module260of the operator terminal200has received an input from the operator to select cosmetic data (YES) in the step S22, the cosmetic data transmitting module251of the operator terminal200transmits the selected cosmetic data to the server10(step S23). The display module260may receive an input to select a plurality of cosmetic data, and the cosmetic data transmitting module251may transmit the plurality of cosmetic data to the server10.

The operator image data transmitting module252of the operator terminal200transmits the image of the operator that the imaging module261of the operator terminal200has imaged and the operator image data on a previously registered image of the operator to the server10(step S24).

The cosmetic data receiving module23of the server10receives cosmetic data transmitted from the operator terminal200. The data extracting module40of the server10searches the skin image data database based on the skin image data that was extracted from the face image data that has been received from the customer terminal100and the cosmetic data received from the operator terminal200and extracts made-up skin image data corresponding to the received cosmetic data (step S25). In the step S25, the data extracting module40retrieves unmade-up image data similar to the skin image data extracted from the face image data and made-up skin image data associated with the cosmetic data.

The data synthesis module41of the server10superimposes the extracted made-up skin image data on the skin image data extracted from the face image data received from the customer terminal100to generate synthesized face image data for this customer (step S26). In the step S26, if the face image data is on an entire face, the data synthesis module41superimposes the made-up skin image data on the corresponding parts or the application parts indicated from the cosmetic data. Furthermore, in the step S26, if the face image data is on a partial face, the data synthesis module41superimposes the made-up skin image data on this part. If the face image data is on a partial face, the data synthesis module41may synthesize face image data on the entire face from partial face image data on a plurality of parts of the face transmitted from one customer and then superimpose the made-up skin image data on the synthesized face image data.

The synthesized data transmitting module24of the server10transmits the synthesized face image data generated by the data synthesis module41in the step S26to the customer terminal100(step S27).

The operator image data transceiving module25of the server10receives the operator image data transmitted from the operator terminal200and transmits this operator image data to the customer terminal100(step S28).

The synthesized data receiving module151of the customer terminal100receives the synthesized face image data transmitted from the server10. The operator image data receiving module152of the customer terminal100receives the operator image data transmitted from the server10. The display module160of the customer terminal100displays the face image, the synthesized face image based on the received synthesized face image data, and the operator image based on the operator image data (step S29).

The screen that the display module160of the customer terminal100displays in the step S29will be explained with reference toFIGS. 11 and 12.

FIG. 11shows a face image, a synthesized face image, and an operator image that the display module260of the customer terminal100displays if the imaging module261of the customer terminal100has imaged the entire face of the customer. The display module260displays a face image410, a synthesized face image500, and an operator image510.FIG. 11shows that the cosmetics600and610are applied to the synthesized face image500. The display module260displays a cosmetic data display window620showing the cosmetics600and610on or around the places where the cosmetics600and610have been applied. The display module260may display a cosmetic data display window620at other places. For example, the cosmetic data display window620may be displayed in a balloon or a blank area to show the cosmetic data being used and the application part in the operator image510. The cosmetic data being used may be displayed when the customer selects the application part. The display module260may shows the cosmetic data being used and the application part by another method. In this embodiment, the display module260shows two kinds of cosmetic data but may be one kind or three or more kinds. The display module260may receive an input of the place to which a cosmetic is applied, from the customer, and may display cosmetic data and made-up skin image data for this place.

FIG. 12shows a partial face image, a synthesized partial face image, and an operator image that the display module260of the customer terminal100displays if the imaging module261of the customer terminal100has imaged the partial face of the customer. The display module260displays a partial face image411, a synthesized partial face image700, and an operator image510.FIG. 12shows that the cosmetic800is applied to one part in the synthesized face image700. The display module260displays a cosmetic data display window620showing cosmetics used on or around the places where the cosmetic800has been applied. The display module260may display a cosmetic data display window620at other places. For example, the cosmetic data display window620may be displayed in a balloon or a blank area to show the cosmetic data being used and the application part in the operator image510. The cosmetic data being used may be displayed when the customer selects the application part. The display module260may shows the cosmetic data being used and the application part by another method. In this embodiment, the display module260shows one kind of cosmetic data but may be two or more kinds. The display module260may receive an input of the place to which a cosmetic is applied, from the customer, and may display cosmetic data and made-up skin image data for this place.

To achieve the means and the functions that are described above, a computer (including a CPU, an information processor, and various terminals) reads and executes a predetermined program. For example, a program is provided in forms recorded in a computer-readable record medium such as a flexible disk, a CD (e.g., CD-ROM), or a DVD (e.g., DVD-ROM, DVD-RAM). In this case, a computer reads a program from the record medium, forwards and stores the program to and in an internal or an external storage, and executes it. For example, the program may be previously recorded in a storage (record medium) such as a magnetic disk, an optical disk, or a magnetic optical disk, and then provided from the storage to a computer through a communication line.

The embodiments of the present invention are described above. However, the present invention is not limited to the above-mentioned embodiments. The effect described in the embodiments of the present invention is only the most preferable effect produced from the present invention. The effects of the present invention are not limited to those described in the embodiments of the present invention.

REFERENCE SIGNS LIST