Patent Description:
Although individuals invest a substantial amount of money in makeup tools and accessories, it can be a challenge to achieve the same results as a makeup professional. In retail establishments, professional beauty advisors commonly assist individuals with recommending cosmetic products to achieve a desired look. However, due to the volume of consumers seeking makeup consultation at times, the availability of beauty advisors may be limited. Although some retail establishments offer the use of electronic kiosks that allow customers to evaluate cosmetic products, it can be difficult for customers to retain makeup information given the number of cosmetic products that a customer may try on during a particular session. Furthermore, some customers may not wish to go through the tedious process of setting up an account at the electronic kiosk for purposes of tracking makeup information. Therefore, there is a need for allowing consumers to efficiently evaluate and track information on cosmetic products for purposes of later purchasing products, participating in a makeup consultation, and so on.

<CIT> discloses techniques for generating virtual makeup products. Virtual makeup products, in this context, may include a predefined effect and a configuration for the effect, wherein the predefined effect includes one or more layered visual filters. In an embodiment, a system in accordance with the present disclosure may acquire makeup product definition data based on characteristics of a real-world makeup product from an entity associated with the real-world product such as a designer, manufacturer, or vendor. Using the makeup product definition data, the system may generate a virtual makeup product includes a configuration of a predefined effect based on the acquired product definition data. The generated virtual makeup product is then passed through a process of internal calibration, internal review, and external review to improve the accuracy of the simulation of the real-world makeup product. In some embodiments, external review is performed by an external reviewing entity (e.g., a designer, manufacturer, or vendor) by using a validation application.

It is an object of the present invention to provide an improved concept for a virtual application of cosmetic effects.

This object is achieved by a method of claim <NUM>, a system of claim <NUM>, or a computer program of claim <NUM>.

In accordance with one embodiment, a makeup evaluation computing device obtains a request from a user to initiate a makeup evaluation session and initiates the makeup evaluation session without obtaining login credentials from the user. The makeup evaluation computing device obtains a digital image of a facial region of the user and generates a user interface displaying at least one cosmetic product. The makeup evaluation computing device obtains from the user a selection of cosmetic products among the one or more displayed cosmetic products and performs virtual application of the selected cosmetic products on the facial region of the user. The makeup evaluation computing device stores the selection of cosmetic products and generates a makeup session packet comprising the digital image of the facial region of the user and stored selection of cosmetic products. The makeup evaluation computing device transmits the makeup session packet to a cloud server and terminates the makeup evaluation session.

Furthermore, a client device obtains a request from a user to retrieve makeup information from a cloud server. The client device obtains a digital image of a facial region of the user and transmits the digital image to initiate a communication session with the cloud server without obtaining login credentials from the user, wherein the cloud server retrieves a makeup session packet by matching the transmitted digital image with one of a plurality of digital images in the cloud server and transmits the makeup session packet comprising a corresponding selection of cosmetic products. The client device receives the makeup session packet and extracts the selection of cosmetic products. The client device generates a user interface displaying the selection of cosmetic products and performs virtual application of the selection of cosmetic products onto the facial region of the user.

Another embodiment comprises a makeup evaluation system that comprises a memory storing instructions and a processor coupled to the memory. The processor is configured by the instructions to obtain a request from a user to initiate a makeup evaluation session and initiate the makeup evaluation session without obtaining login credentials from the user. The processor is further configured to obtain a digital image of a facial region of the user and generate a user interface displaying at least one cosmetic product. The processor is further configured to obtain from the user a selection of cosmetic products among the one or more displayed cosmetic products and perform virtual application of the selected cosmetic products on the facial region of the user. The processor is further configured to store the selection of cosmetic products and generate a makeup session packet comprising the digital image of the facial region of the user and stored selection of cosmetic products. The processor is further configured to transmit the makeup session packet to a cloud server and terminate the makeup evaluation session.

This embodiment comprises a system that comprises a memory storing instructions and a processor coupled to the memory. The processor is configured by the instructions to obtain a request from a user to retrieve makeup information from a cloud server. The processor is further configured to obtain a digital image of a facial region of the user and transmit the digital image to initiate a communication session with the cloud server without obtaining login credentials from the user, wherein the cloud server retrieves a makeup session packet by matching the transmitted digital image with one of a plurality of digital images in the cloud server and transmits the makeup session packet comprising a corresponding selection of cosmetic products. The processor is further configured to receive the makeup session packet and extracting the selection of cosmetic products and generate a user interface displaying the selection of cosmetic products. The processor is further configured to perform virtual application of the selection of cosmetic products onto the facial region of the user.

Various aspects of the disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, with emphasis instead being placed upon clearly illustrating the principles of the present disclosure.

Various embodiments are disclosed for allowing individuals to receive and evaluate makeup product recommendations, where the individuals are able to later retrieve information relating to makeup products of interest through the use of a digital image, where such information may be utilized, for example, to purchase makeup products or to participate in a makeup consultation session with an available makeup professional. Notably, various embodiments are disclosed that do not require the use of login credentials (e.g., username and password), thereby allowing individuals to access information stored in the cloud without having to submit such information as contact information, e-mail address, and so on that individuals may not wish to share.

A description of a system for performing virtual application of cosmetic products based on facial identification and corresponding makeup information is now described followed by a discussion of the operation of the components within the system. <FIG> is a block diagram of a networked environment that includes a makeup evaluation computing device <NUM>, a cloud server device <NUM>, and a client device <NUM>. The makeup evaluation computing device <NUM>, cloud server device <NUM>, and the client device <NUM> are communicatively coupled via a network <NUM> such as, for example, the Internet, intranets, extranets, wide area networks (WANs), local area networks (LANs), wired networks, wireless networks, or other suitable networks, etc., or any combination of two or more such networks.

Both the makeup evaluation computing device <NUM> and the client device <NUM> are embodied as a computing device such as, but not limited to, a smartphone, a tablet computing device, a laptop, and so on. The cloud server device <NUM> may comprise a server computer or any other system providing computing capability. Alternatively, the cloud server device <NUM> may employ a plurality of computing devices that can be arranged, for example, in one or more server banks, computer banks or other arrangements. Such computing devices can be located in a single installation or can be distributed among different geographical locations.

The makeup evaluation computing device <NUM> may be installed in a retail establishment and is utilized by a user to receive and evaluate makeup product recommendations generated by the makeup evaluation computing device <NUM>. A makeup evaluation service <NUM> executes on a processor of the makeup evaluation computing device <NUM> thereby causing the makeup evaluation computing device <NUM> to perform the operations/functions for implementing the features disclosed herein. The makeup evaluation service <NUM> includes a session manager <NUM>, a user interface (Ul) generator <NUM>, a virtual makeup applicator <NUM>, and a packet generator <NUM>.

The session manager <NUM> is configured to obtain a request from a user to initiate a makeup evaluation session. The session manager <NUM> initiates the makeup evaluation session without obtaining login credentials from the user. Rather than utilizing traditional login credentials (e.g., username and password) to initiate the makeup session, the session manager <NUM> obtains a digital image of a facial region of the user and utilizes the digital image to initiate the makeup evaluation session. One advantage of bypassing the use of login credentials is that the user does not have to spend time setting up an account on the makeup evaluation computing device <NUM> to access content on the cloud server device <NUM>. Furthermore, the user does not have to submit such information as contact information, e-mail address, and so on that individuals may wish to keep private.

As one of ordinary skill will appreciate, the digital image may be encoded in any of a number of formats including, but not limited to, JPEG (Joint Photographic Experts Group) files, TIFF (Tagged Image File Format) files, PNG (Portable Network Graphics) files, GIF (Graphics Interchange Format) files, BMP (bitmap) files or any number of other digital formats. Alternatively, the digital image may be derived from a still image of a video encoded in formats including, but not limited to, Motion Picture Experts Group (MPEG)-<NUM>, MPEG-<NUM>, MPEG-<NUM>, H. <NUM>, Third Generation Partnership Project (3GPP), 3GPP-<NUM>, Standard-Definition Video (SD-Video), High-Definition Video (HD-Video), Digital Versatile Disc (DVD) multimedia, Video Compact Disc (VCD) multimedia, High-Definition Digital Versatile Disc (HD-DVD) multimedia, Digital Television Video / High-definition Digital Television (DTV/HDTV) multimedia, Audio Video Interleave (AVI), Digital Video (DV), QuickTime (QT) file, Windows Media Video (WMV), Advanced System Format (ASF), Real Media (RM), Flash Media (FLV), an MPEG Audio Layer III (MP3), an MPEG Audio Layer II (MP2), Waveform Audio Format (WAV), Windows Media Audio (WMA), <NUM> degree video, 3D scan model, or any number of other digital formats.

The UI generator <NUM> is configured to generate a user interface displaying one or more cosmetic products. The UI generator <NUM> obtains from the user a selection of cosmetic products among the one or more displayed cosmetic products. The virtual makeup applicator <NUM> is configured to perform virtual application of the selected cosmetic products on the facial region of the user, thereby allowing the user to evaluate the application of the selected cosmetic products. The packet generator <NUM> is configured to generate a makeup session packet <NUM> comprising the digital image <NUM> of the facial region of the user and the stored selection of cosmetic products <NUM>. The packet generator <NUM> transmits the makeup session packet <NUM> to the cloud server device <NUM>. A data management service <NUM> executes on a processor of the cloud server device <NUM> and stores the makeup session packet <NUM> in a data store <NUM>.

A virtual makeup application <NUM> executes on a processor of the client device <NUM> thereby causing the client device <NUM> to perform the operations/functions for implementing the features disclosed herein. The virtual makeup applicator <NUM> is configured to obtain a request from a user to retrieve makeup information from the cloud server device <NUM>. The virtual makeup application <NUM> initiates a communication session with the cloud server device <NUM> without obtaining login credentials from the user. As discussed above, one advantage of bypassing the use of login credentials is that the user does not have to spend time setting up an account to access content on the cloud server device <NUM>. Furthermore, the user does not have to submit such information as contact information, e-mail address, and so on that individuals may wish to keep private.

The virtual makeup application <NUM> obtains a digital image of a facial region of the user and transmits the digital image to the cloud server device <NUM>, wherein the cloud server device <NUM> retrieves a makeup session packet <NUM> comprising a matching digital image of the facial region of the user and a corresponding selection of cosmetic products. The cloud server device <NUM> sends the makeup session packet <NUM> to the client device <NUM>, and the virtual makeup application <NUM> generates a user interface displaying the selection of cosmetic products specified in the makeup session packet <NUM>. The virtual makeup application <NUM> performs virtual application of the selection of cosmetic products onto the facial region of the user.

<FIG> illustrates a schematic block diagram for each of the makeup evaluation computing device <NUM>, the cloud server device <NUM>, and the client device <NUM> in <FIG>. Each of these computing devices <NUM>, <NUM>, <NUM> may be embodied in any one of a wide variety of wired and/or wireless computing devices, such as a desktop computer, portable computer, dedicated server computer, multiprocessor computing device, smart phone, tablet, and so forth. As shown in <FIG>, each of these computing devices <NUM>, <NUM>, <NUM> comprises memory <NUM>, a processing device <NUM>, a number of input/output interfaces <NUM>, a network interface <NUM>, a display <NUM>, a peripheral interface <NUM>, and mass storage <NUM>, wherein each of these components are connected across a local data bus <NUM>.

The processing device <NUM> may include any custom made or commercially available processor, a central processing unit (CPU) or an auxiliary processor among several processors associated with the computing device, a semiconductor based microprocessor (in the form of a microchip), a macroprocessor, one or more application specific integrated circuits (ASICs), a plurality of suitably configured digital logic gates, and other well known electrical configurations comprising discrete elements both individually and in various combinations to coordinate the overall operation of the computing system.

The memory <NUM> may include any one of a combination of volatile memory elements (e.g., random-access memory (RAM, such as DRAM, and SRAM, etc.)) and nonvolatile memory elements (e.g., ROM, hard drive, tape, CDROM, etc.). The memory <NUM> typically comprises a native operating system <NUM>, one or more native applications, emulation systems, or emulated applications for any of a variety of operating systems and/or emulated hardware platforms, emulated operating systems, etc. For example, the applications may include application specific software which may comprise some or all the components of the computing devices <NUM>, <NUM>, <NUM> depicted in <FIG>. In accordance with such embodiments, the components are stored in memory <NUM> and executed by the processing device <NUM>, thereby causing the processing device <NUM> to perform the operations/functions relating to the features disclosed herein. One of ordinary skill in the art will appreciate that the memory <NUM> can, and typically will, comprise other components which have been omitted for purposes of brevity. For some embodiments, the components in the computing devices <NUM>, <NUM>, <NUM> may be implemented by hardware and/or software.

Input/output interfaces <NUM> provide any number of interfaces for the input and output of data. For example, where the computing device comprises a personal computer, these components may interface with one or more user input/output interfaces <NUM>, which may comprise a keyboard or a mouse, as shown in <FIG>. The display <NUM> may comprise a computer monitor, a plasma screen for a PC, a liquid crystal display (LCD) on a hand held device, a touchscreen, or other display device.

In the context of this disclosure, a non-transitory computer-readable medium stores programs for use by or in connection with an instruction execution system, apparatus, or device. More specific examples of a computer-readable medium may include by way of example and without limitation: a portable computer diskette, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM, EEPROM, or Flash memory), and a portable compact disc read-only memory (CDROM) (optical).

Reference is made to <FIG>, which is a flowchart <NUM> that shows the interaction between the various components in the networked environment of <FIG> for performing virtual application of cosmetic products based on facial identification and corresponding makeup information. It is understood that the flowchart <NUM> of <FIG> provides merely an example of the different types of functional arrangements that may be employed to implement the operation of the various components of the computing devices <NUM>, <NUM>, <NUM> depicted in <FIG>. As an alternative, the flowchart <NUM> of <FIG> may be viewed as depicting an example of steps of a method implemented in the computing devices <NUM>, <NUM>, <NUM> according to one or more embodiments.

Although the flowchart <NUM> of <FIG> shows a specific order of execution, it is understood that the order of execution may differ from that which is depicted. For example, the order of execution of two or more blocks may be scrambled relative to the order shown. Also, two or more blocks shown in succession in <FIG> may be executed concurrently or with partial concurrence. It is understood that all such variations are within the scope of the present disclosure.

At block <NUM>, the makeup evaluation computing device <NUM> initiates a makeup evaluation session based on input from a user. For some embodiments, the makeup evaluation computing device <NUM> may be installed in a retail establishment. At block <NUM>, the makeup evaluation computing device <NUM> obtains a digital image of the facial region of the user and provides cosmetic products. The cosmetic products are displayed on a user interface generated by the makeup evaluation computing device <NUM>, where the cosmetic products may be displayed based on facial features and attributes depicted in the digital image. At block <NUM>, the makeup evaluation computing device <NUM> obtains selections of one or more of the cosmetic products and performs virtual application of the selected cosmetic products onto the digital image of the user.

At block <NUM>, the makeup evaluation computing device <NUM> generates a makeup session packet <NUM> (<FIG>) by bundling the digital image with the cosmetic products selected by the user. Other information such as price, ingredients, manufacturing location, usage about the cosmetic products, and so on may also be bundled into the makeup session packet. The makeup evaluation computing device <NUM> then uploads the makeup session packet to the cloud server device <NUM>. At block <NUM>, the cloud server device <NUM> stores the makeup session packet <NUM> in a data store <NUM> (<FIG>).

At block <NUM>, the makeup evaluation computing device <NUM> terminates the makeup evaluation session. The termination process may comprise deleting the digital image of the user and the stored selection of cosmetic products selected by the user from a local storage of the makeup evaluation computing device <NUM>. In this regard, the digital image and the selection of cosmetic products are only stored temporarily by the makeup evaluation computing device <NUM>.

Later, the user may wish to retrieve information relating to the cosmetic products selected earlier by the user while visiting the retail establishment. In exemplary embodiments, the user accesses this information using a client device <NUM> such as the user's smartphone, laptop, or other computing device communicatively coupled to the cloud server device <NUM>. At block <NUM>, the client device <NUM> obtains a digital image of the facial region of the user. At block <NUM>, the client device <NUM> transmits the digital image to the cloud server device <NUM> to initiate a communication session with the cloud server device <NUM>. Note that the client device <NUM> initiates the communication session without the use of any kind of login credentials (e.g., username and password) associated with the user.

At block <NUM>, the cloud server device <NUM> analyzes the digital image uploaded by the client device <NUM> and searches the data store <NUM> and compares the uploaded digital image to digital images in each of the makeup session packets <NUM> in the data store <NUM>. The cloud server device <NUM> retrieves a closest matching digital image in the data store <NUM> and transmits the corresponding makeup session packet <NUM> to the client device <NUM>.

At block <NUM>, the client device <NUM> parses the makeup session packet <NUM> downloaded from the cloud server device <NUM> and obtains the cosmetic products embedded in the makeup session packet <NUM>. The client device <NUM> generates a user interface that displays a listing of the cosmetic products in the order they were selected earlier by the user using the makeup evaluation computing device <NUM> at the retail establishment. The client device <NUM> performs virtual application of the cosmetic products obtained from the downloaded the makeup session packet <NUM>.

For some embodiments, the client device <NUM> may also determining a grouping of beauty advisors currently online and display a listing of beauty advisors based on the cosmetic products specified in the makeup session packet <NUM> downloaded from the cloud server device <NUM>. Specifically, the client device <NUM> may be configured to retrieve profile information for each of the online beauty advisors and identify beauty advisors who have prior experience working with one or more of the cosmetic products specified in the makeup session packet <NUM>. The client device <NUM> displays a list of suitable beauty advisors to the user (block <NUM>).

The user may then initiate a makeup consultation session with a beauty advisor to receive further cosmetic products or to further refine application of the cosmetic products previously selected by the user. For some embodiments, the user may be prompted to enter login credentials by the client device <NUM> in order to provide the beauty advisor with relevant information about the user for purposes of facilitating the makeup consultation session. Additional cosmetic products and refinement of the previously selected cosmetic products made during the makeup consultation session may then be stored in connection with the user's login credentials. Thereafter, the process in <FIG> ends.

Claim 1:
A method comprising:
a first method implemented in a first computing device (<NUM>), the first method comprising:
obtaining a request from a user to initiate a makeup evaluation session;
initiating (<NUM>) the makeup evaluation session without obtaining login credentials from the user;
obtaining (<NUM>) a first digital image of a facial region of the user;
generating a first user interface displaying at least one cosmetic product;
obtaining (<NUM>) from the user a selection of cosmetic products among the at least one displayed cosmetic products;
performing (<NUM>) virtual application of the selected cosmetic products on the facial region of the user;
storing the selection of cosmetic products;
generating (<NUM>) a makeup session packet comprising the first digital image of the facial region of the user and stored selection of cosmetic products;
transmitting (<NUM>) the makeup session packet to a cloud server (<NUM>); and
terminating (<NUM>) the makeup evaluation session; and
a second method implemented in a second computing device (<NUM>), the second method comprising:
obtaining a request from a user to retrieve makeup information from the cloud server (<NUM>);
obtaining (<NUM>) a second digital image of a facial region of the user;
transmitting (<NUM>) the second digital image to initiate a communication session with the cloud server (<NUM>) without obtaining login credentials from the user, wherein the cloud server (<NUM>) retrieves (<NUM>) a makeup session packet by matching the second transmitted digital image with one of a plurality of first digital images in the cloud server (<NUM>) and transmits (<NUM>) the makeup session packet comprising a corresponding selection of cosmetic products;
receiving the makeup session packet and extracting the selection of cosmetic products;
generating a second user interface displaying the selection of cosmetic products; and
performing (<NUM>) a virtual application of the selection of cosmetic products onto the facial region of the user.