SYSTEMS AND METHODS FOR MULTI-TIERED GENERATION OF A FACE CHART

A computing device obtains an image depicting an image of a user's face. The computing device identifies one or more regions in the image depicting skin of the user and generates a skin mask. A skin tone of the user's face is predicted and the skin mask is populated according to the predicted skin tone. The computing device defines feature points corresponding to facial features on the user's face and extracts pre-defined facial patterns matching facial features depicted in the image. The extracted pre-defined facial patterns are inserted into the skin mask based on the feature points and a hair mask identifying one or more regions depicting hair of the user is generated. The computing device extracts a hair region depicted in the image of the user based on the hair mask and inserts the hair region on top of the skin mask to generate a face chart.

TECHNICAL FIELD

The present disclosure generally relates to systems and methods for multi-tiered generation of a face chart using, for example, machine-learning techniques.

SUMMARY

In accordance with one embodiment, a computing device obtains an image depicting an image of a user's face. The computing device identifies one or more regions in the image depicting skin of the user and generates a skin mask. The computing device predicts a skin tone of the user's face depicted in the image and populates the skin mask according to the predicted skin tone. The computing device defines feature points corresponding to facial features on the user's face depicted in the image and extracts pre-defined facial patterns matching facial features depicted in the image. The computing device inserts the extracted pre-defined facial patterns into the skin mask based on the feature points and generates a hair mask identifying one or more regions in the image depicting hair of the user. The computing device extracts a hair region depicted in the image of the user based on the hair mask and inserts the hair region on top of the

Another embodiment is a system that comprises a memory storing instructions and a processor coupled to the memory. The processor is configured by the instructions to obtain an image depicting an image of a user's face. The processor is further configured to identify one or more regions in the image depicting skin of the user and generate a skin mask. The processor is further configured to predict a skin tone of the user's face depicted in the image and populate the skin mask according to the predicted skin tone. The processor is further configured to define feature points corresponding to facial features on the user's face depicted in the image and extract pre-defined facial patterns matching facial features depicted in the image. The processor is further configured to insert the extracted pre-defined facial patterns into the skin mask based on the feature points and generate a hair mask identifying one or more regions in the image depicting hair of the user. The processor is further configured to extract a hair region depicted in the image of the user based on the hair mask and insert the hair region on top of the skin mask to generate a face chart.

Another embodiment is a non-transitory computer-readable storage medium storing instructions to be implemented by a computing device. The computing device comprises a processor, wherein the instructions, when executed by the processor, cause the computing device to obtain an image depicting an image of a user's face. The processor is further configured by the instructions to identify one or more regions in the image depicting skin of the user and generate a skin mask. The processor is further configured by the instructions to predict a skin tone of the user's face depicted in the image and populate the skin mask according to the predicted skin tone. The processor is further configured by the instructions to define feature points corresponding to facial features on the user's face depicted in the image and extract pre-defined facial patterns matching facial features depicted in the image. The processor is further configured to insert the extracted pre-defined facial patterns into the skin mask based on the feature points and generate a hair mask identifying one or more regions in the image depicting hair of the user. The processor is further configured to extract a hair region depicted in the image of the user based on the hair mask and insert the hair region on top of the skin mask to generate a face chart.

Other systems, methods, features, and advantages of the present disclosure will be apparent to one skilled in the art upon examining the following drawings and detailed description. It is intended that all such additional systems, methods, features, and advantages be included within this description, be within the scope of the present disclosure, and be protected by the accompanying claims.

DETAILED DESCRIPTION

The subject disclosure is now described with reference to the drawings, where like reference numerals are used to refer to like elements throughout the following description. Other aspects, advantages, and novel features of the disclosed subject matter will become apparent from the following detailed description and corresponding drawings.

The present disclosure is directed to systems and methods for multi-tiered generation of face charts that capture characteristics of the individual's facial features that are more accurate in comparison to conventional configurations, thereby facilitating selection and application of the most suitable cosmetic products for the individual. Face charts may be used by makeup artists to design looks using various cosmetics based on the specific characteristics of a user's face. Therefore, a face chart that accurately captures characteristics of a user's face is essential. A description of a system for performing multi-tiered generation of a face chart is described followed by a discussion of the operation of the components within the system.

FIG.1is a block diagram of a computing device102in which the embodiments disclosed herein may be implemented. The computing device102may comprise one or more processors that execute machine executable instructions to perform the features described herein. For example, the computing device102may be embodied as a computing device such as, but not limited to, a smartphone, a tablet-computing device, a laptop, and so on.

A face chart constructor104is executed by a processor of the computing device102and includes an image capture module105, a facial feature analyzer106, and a layer aggregator116. The facial feature analyzer106generates different layers of the face chart where each layer captures characteristics relating to different aspects of the user's face, as described in more detail below. The facial feature analyzer106includes a skin mask module108, a hair mask module110, a skin tone predictor112, and a facial features module114. The layer aggregator116is configured to combine all the layers generated by the facial feature analyzer106and generate a final face chart118of the user.

The image capture module105is configured to obtain digital images101of a user's face. For some embodiments, the image capture module105is configured to cause a camera of the computing device102to capture an image101or a video of the user of the computing device102.FIG.4illustrates an example user interface402with a virtual mirror feature generated on a display of the computing device102where a digital image of a user is shown in the user interface402. For some embodiments, the computing device102is equipped with a front facing camera that captures an image of the user for multi-tiered generation of a face chart for the user.

Referring back toFIG.1, the computing device102may also be equipped with the capability to connect to the Internet, and the image capture module105may be configured to obtain an image or video of the user from another device or server. This feature is used, for example, when the skin mask module108, the hair mask module110, and/or the skin tone predictor112perform corresponding functions by executing a machine-learning algorithm based on other images of the user.

The images obtained by the image capture module105may be encoded in any of a number of formats including, but not limited to, JPEG (Joint Photographic Experts Group) files, TIFF (Tagged Image File Format) files, PNG (Portable Network Graphics) files, GIF (Graphics Interchange Format) files, BMP (bitmap) files or any number of other digital formats. The video may be encoded in formats including, but not limited to, Motion Picture Experts Group (MPEG)-1, MPEG-2, MPEG-4, H.264, Third Generation Partnership Project (3GPP), 3GPP-2, Standard-Definition Video (SD-Video), High-Definition Video (HD-Video), Digital Versatile Disc (DVD) multimedia, Video Compact Disc (VCD) multimedia, High-Definition Digital Versatile Disc (HD-DVD) multimedia, Digital Television Video/High-definition Digital Television (DTV/HDTV) multimedia, Audio Video Interleave (AVI), Digital Video (DV), QuickTime (QT) file, Windows Media Video (WMV), Advanced System Format (ASF), Real Media (RM), Flash Media (FLV), an MPEG Audio Layer III (MP3), an MPEG Audio Layer II (MP2), Waveform Audio Format (WAV), Windows Media Audio (WMA), 360 degree video, 3D scan model, or any number of other digital formats.

With reference toFIG.5, the skin mask module108executing in the computing device102ofFIG.1is configured to receive the digital image101captured by the image capture module105(FIG.1), identify one or more regions in the image101depicting the user's skin and generate a skin mask502based on the identified regions. The skin mask502is used to differentiate between the user's skin, the user's hair, the background of the image, and so on when applying cosmetic effects to the user's face. For some embodiments, the layer aggregator116receives the skin mask502generated by the skin mask module108and generates a first layer of the face chart comprising the skin mask502.

With reference toFIG.6, the hair mask module110executing in the computing device102ofFIG.1is configured to receive the digital image101captured by the image capture module105(FIG.1) and generate a hair mask602that identifies one or more regions in the image depicting the user's hair. The layer aggregator116inserts the generated hair mask602as a second layer into the face chart on top of the skin mask generated earlier. For some embodiments, the hair mask module110applies a machine-learning algorithm to other images of the user to identify more accurately the one or more regions depicting the user's hair.

With reference toFIG.7, the skin tone predictor112executing in the computing device102ofFIG.1is configured to receive the digital image101captured by the image capture module105(FIG.1) and predict a skin tone of the user's face depicted in the image of the user. The skin tone predictor112then populates the face chart according to the predicted skin tone. For some embodiments, the skin tone predictor112applies a machine-learning algorithm to other images of the user as well as to images of other individuals to obtain a more accurate prediction of the user's skin tone.

With reference toFIG.8, the facial features module114executing in the computing device102ofFIG.1is configured to receive the digital image101captured by the image capture module105(FIG.1) and extract pre-defined facial patterns matching facial features depicted in the image101of the user's face based on the feature points. The facial features module114inserts the extracted pre-defined facial patterns into the face chart based on the feature points.

In the example shown inFIG.8, pre-defined nose types802are compared to the features points of the user's nose depicted in the image101. The pre-defined nose type802that most closely matches the user's nose is then inserted in the face chart as an estimated nose feature. As another example, pre-defined eye types are compared to the features points of the user's eye depicted in the image101. The pre-defined eye type that most closely matches the user's eye is then inserted in the face chart as an estimated eye feature. As yet another example, pre-defined mouth types are compared to the features points of the user's mouth depicted in the image101. The pre-defined mouth type that most closely matches the user's mouth is then inserted in the face chart as an estimated mouth feature.

FIG.2illustrates a schematic block diagram of the computing device102inFIG.1. The computing device102may be embodied as a desktop computer, portable computer, dedicated server computer, multiprocessor computing device, smart phone, tablet, and so forth. As shown inFIG.2, the computing device102comprises memory214, a processing device202, a number of input/output interfaces204, a network interface206, a display208, a peripheral interface211, and mass storage226, wherein each of these components are connected across a local data bus210.

The processing device202may include a custom made processor, a central processing unit (CPU), or an auxiliary processor among several processors associated with the computing device102, a semiconductor based microprocessor (in the form of a microchip), a macroprocessor, one or more application specific integrated circuits (ASICs), a plurality of suitably configured digital logic gates, and so forth.

The memory214may include one or a combination of volatile memory elements (e.g., random-access memory (RAM) such as DRAM and SRAM) and nonvolatile memory elements (e.g., ROM, hard drive, tape, CDROM). The memory214typically comprises a native operating system216, one or more native applications, emulation systems, or emulated applications for any of a variety of operating systems and/or emulated hardware platforms, emulated operating systems, etc. For example, the applications may include application specific software that may comprise some or all the components of the computing device102displayed inFIG.1.

In accordance with such embodiments, the components are stored in memory214and executed by the processing device202, thereby causing the processing device202to perform the operations/functions disclosed herein. For some embodiments, the components in the computing device102may be implemented by hardware and/or software.

Input/output interfaces204provide interfaces for the input and output of data. For example, where the computing device102comprises a personal computer, these components may interface with one or more input/output interfaces204, which may comprise a keyboard or a mouse, as shown inFIG.2. The display208may comprise a computer monitor, a plasma screen for a PC, a liquid crystal display (LCD) on a hand held device, a touchscreen, or other display device.

In the context of this disclosure, a non-transitory computer-readable medium stores programs for use by or in connection with an instruction execution system, apparatus, or device. More specific examples of a computer-readable medium may include by way of example and without limitation: a portable computer diskette, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM, EEPROM, or Flash memory), and a portable compact disc read-only memory (CDROM) (optical).

Reference is made toFIG.3, which is a flowchart300in accordance with various embodiments for performing multi-tiered generation of a face chart, where the operations are performed by the computing device102ofFIG.1. It is understood that the flowchart300ofFIG.3provides merely an example of the different types of functional arrangements that may be employed to implement the operation of the various components of the computing device102. As an alternative, the flowchart300ofFIG.3may be viewed as depicting an example of steps of a method implemented in the computing device102according to one or more embodiments.

Although the flowchart300ofFIG.3shows a specific order of execution, it is understood that the order of execution may differ from that which is displayed. For example, the order of execution of two or more blocks may be scrambled relative to the order shown. In addition, two or more blocks shown in succession inFIG.3may be executed concurrently or with partial concurrence. It is understood that all such variations are within the scope of the present disclosure.

At block310, the computing device102obtains an image depicting an image of a user's face. At block320, the computing device102identifies one or more regions in the image depicting skin of the user and generates a skin mask. For some embodiments, the computing device102identifies the one or more regions in the image depicting the user's skin and generates the skin mask by executing a machine-learning algorithm based on other images of the user.

At block330, the computing device102predicts a skin tone of the user's face depicted in the image and populates the skin mask according to the predicted skin tone. For some embodiments, the computing device102predicts the skin tone of the user's face depicted in the image of the user by executing a machine-learning algorithm based on other images of the user and other individuals.

At block340, the computing device102defines feature points corresponding to facial features on the user's face depicted in the image. For some embodiments, the computing device102defines the feature points corresponding to the facial features on the user's face depicted in the image is performed by utilizing a convolutional neural network.

At block350, the computing device102extracts pre-defined facial patterns matching facial features depicted in the image. For some embodiments, the pre-defined facial patterns may include, but are not limited to, an eye, a mouth, a nose, or an eyebrow. At block360, the computing device102inserts the extracted pre-defined facial patterns into the skin mask based on the feature points.

At block370, the computing device102generates a hair mask identifying one or more regions in the image depicting hair of the user. For some embodiments, the computing device102generates the hair mask identifying the one or more regions in the image depicting the user's hair by executing a machine-learning algorithm based on other images of the user.

At block380, the computing device102extracts a hair region depicted in the image of the user based on the hair mask and inserts the hair region on top of the skin mask to generate a face chart. For some embodiments, the computing device102inserts the hair region on top of the skin mask to generate the face chart by extracting the hair region depicted in the image of the user and inserting the extracted hair region on top of the skin mask. As an alternative, the computing device102inserts the hair region on top of the skin mask to generate the face chart by inserting a sketch drawing of the user's hair on top of the skin mask. For some embodiments, the computing device102generates the face chart by inserting a background into the face chart or superimposing the skin mask on the background. For some embodiments, the background is extracted from the image of the user's face. Thereafter, the process inFIG.3ends.