SYSTEMS AND METHODS FOR ADJUSTING LIGHTING INTENSITY OF A FACE CHART

A computing device obtains an image depicting a face of a user. The computing device identifies facial features in the image and extracts characteristics of the facial features in the image. The computing device generates a two-dimensional (2D) face chart based on the facial feature characteristics. The computing device predicts a skin tone of the user's face depicted in the image of the user and changes color in a color map of a predefined three-dimensional (3D) model based on the predicted skin tone. The computing device selects a predefined environment map based on characteristics in the image depicting the face of the user and generates a target face image based on the predefined 3D model.

TECHNICAL FIELD

The present disclosure generally relates to systems and methods for adjusting the lighting intensity of face charts.

SUMMARY

In accordance with one embodiment, a computing device obtains an image depicting a face of a user. The computing device identifies facial features in the image and extracts characteristics of the facial features in the image. The computing device generates a two-dimensional (2D) face chart based on the facial feature characteristics. The computing device predicts a skin tone of the user's face depicted in the image of the user and changes color in a color map of a predefined three-dimensional (3D) model based on the predicted skin tone. The computing device selects a predefined environment map based on characteristics in the image depicting the face of the user. The computing device generates a target face image based on the predefined 3D model.

Another embodiment is a system that comprises a memory storing instructions and a processor coupled to the memory. The processor is configured by the instructions to obtain an image depicting a face of a user. The processor is further configured to identify facial features in the image and extract characteristics of the facial features in the image. The processor is further configured to generate a two-dimensional (2D) face chart based on the facial feature characteristics. The processor is further configured to predict a skin tone of the user's face depicted in the image of the user and change color in a color map of a predefined three-dimensional (3D) model based on the predicted skin tone. The processor is further configured to select a predefined environment map based on characteristics in the image depicting the face of the user. The processor is further configured to generate a target face image based on the predefined 3D model.

Another embodiment is a non-transitory computer-readable storage medium storing instructions to be implemented by a computing device. The computing device comprises a processor, wherein the instructions, when executed by the processor, cause the computing device to obtain an image depicting a face of a user. The processor is further configured by the instructions to identify facial features in the image and extract characteristics of the facial features in the image. The processor is further configured by the instructions to generate a two-dimensional (2D) face chart based on the facial feature characteristics. The processor is further configured by the instructions to predict a skin tone of the user's face depicted in the image of the user and change color in a color map of a predefined three-dimensional (3D) model based on the predicted skin tone. The processor is further configured by the instructions to select a predefined environment map based on characteristics in the image depicting the face of the user. The processor is further configured by the instructions to generate a target face image based on the predefined 3D model.

Other systems, methods, features, and advantages of the present disclosure will be apparent to one skilled in the art upon examining the following drawings and detailed description. It is intended that all such additional systems, methods, features, and advantages be included within this description, be within the scope of the present disclosure, and be protected by the accompanying claims.

DETAILED DESCRIPTION

The subject disclosure is now described with reference to the drawings, where like reference numerals are used to refer to like elements throughout the following description. Other aspects, advantages, and novel features of the disclosed subject matter will become apparent from the following detailed description and corresponding drawings.

Embodiments are disclosed for generating more realistic three-dimensional (3D) face charts depicting a user's facial features, thereby facilitating the selection and application of cosmetic products. A description of a system for adjusting the lighting intensity of a 2Dface chart to enhance a stereoscopic effect of the 2D face chart is described followed by a discussion of the operation of the components within the system.FIG.1is a block diagram of a computing device102in which the embodiments disclosed herein may be implemented. The computing device102may comprise one or more processors that execute machine executable instructions to perform the features described herein. For example, the computing device102may be embodied as a computing device such as, but not limited to, a smartphone, a tablet-computing device, a laptop, and so on.

A face chart application104executes on a processor of the computing device102and includes an image capture module106, a two-dimensional (2D) face chart generator108, a lighting analyzer110, a 3D model retriever112, a 3D model module113, and an image editor116. The image capture module106is configured to obtain digital images of a user's facial region and displays the user's face on a display of the computing device102. The computing device102may also be equipped with the capability to connect to the Internet, and the image capture module106may be configured to obtain an image or video of the user from another device or server.

The images obtained by the image capture module106may be encoded in any of a number of formats including, but not limited to, JPEG (Joint Photographic Experts Group) files, TIFF (Tagged Image File Format) files, PNG (Portable Network Graphics) files, GIF (Graphics Interchange Format) files, BMP (bitmap) files or any number of other digital formats. The video may be encoded in formats including, but not limited to, Motion Picture Experts Group (MPEG)-1, MPEG-2, MPEG-4, H.264, Third Generation Partnership Project (3GPP), 3GPP-2, Standard-Definition Video (SD-Video), High-Definition Video (HD-Video), Digital Versatile Disc (DVD) multimedia, Video Compact Disc (VCD) multimedia, High-Definition Digital Versatile Disc (HD-DVD) multimedia, Digital Television Video/High-definition Digital Television (DTV/HDTV) multimedia, Audio Video Interleave (AVI), Digital Video (DV), QuickTime (QT) file, Windows Media Video (WMV), Advanced System Format (ASF), Real Media (RM), Flash Media (FLV), an MPEG Audio Layer III (MP3), an MPEG Audio Layer II (MP2), Waveform Audio Format (WAV), Windows Media Audio (WMA), 360 degree video, 3D scan model, or any number of other digital formats.

The 2D face chart generator108is executed by the processor of the computing device102to identify facial features in the image obtained by the image capture module106and extract characteristics of the facial features in the image. The characteristics may comprise the size and shape of each facial feature depicted in the image, the overall size and shape of the user's face, contouring information of the user's face, and so on. The 2D face chart generator108generates a 2D face chart based on these facial feature characteristics, where the 2D face chart depicts the user's facial features. The 2D face chart generator108is further configured to predict the skin tone of the user's face depicted in the image.

A 3D model retriever112accesses a data store118containing predefined 3D models120and retrieves a 3D model120that most closely matches the user's face based on the facial characteristics. The 2D face chart generator108is configured to change or adjust color in a color map of a predefined 3D model based on the predicted skin tone, where the color map may comprise, for example, an albedo map.

The lighting analyzer110is executed to analyze the lighting environment depicted in the image of the user's face and to obtain lighting characteristics of the user's face. Such lighting characteristics may include the lighting intensity of each pixel in the facial region of the user and reflects such attributes as the positioning of one or more light sources relative to the user in the image. For some embodiments, the lighting analyzer110is configured to select a predefined environment map based on characteristics of the image. Such characteristics may comprise, for example, an average value of luminance, chrominance, chroma, color temperature, and/or contrast levels in the image. The environment map may comprise, for example, information relating to environmental lighting, predefined environmental lighting associated with different events, a high dynamic range image (HDRI) map, and so on. The events may include, for example, indoor events and outdoor events.

The 3D model module113is executed to generate a target face image based on the predefined 3D model. For some embodiments, the 3D module113superimposes the target face image onto the 2D face chart based on locations of facial features in the target face image. For some embodiments, the 3D module113outputs a final 2D face chart with adjusted lighting. The 3D model module113includes a lighting adjuster114configured to generate one or more lighting effects on the 2D face chart based on the lighting characteristics of the user's face depicted in the image, thereby enhancing a stereoscopic effect of the 2D face chart. In particular, the lighting adjuster114is configured to update the predefined 3D model using the environment map to generate a target face image with stereoscopic, lighting, and shadow effects. For some embodiments, the 3D model module113generates a final 2D face chart by superimposing the target face image onto the 2D face chart based on locations of facial features in the target face image. The image editor116is executed to perform virtual application of one or more cosmetic products on the user's face utilizing the final2D face chart depicting the one or more lighting effects applied by the lighting adjuster114in the 3D model module113.

FIG.2illustrates a schematic block diagram of the computing device102inFIG.1. The computing device102may be embodied as a desktop computer, portable computer, dedicated server computer, multiprocessor computing device, smart phone, tablet, and so forth. As shown inFIG.2, the computing device102comprises memory214, a processing device202, a number of input/output interfaces204, a network interface206, a display208, a peripheral interface211, and mass storage226, wherein each of these components are connected across a local data bus210.

The processing device202may include a custom made processor, a central processing unit (CPU), or an auxiliary processor among several processors associated with the computing device102, a semiconductor based microprocessor (in the form of a microchip), a macroprocessor, one or more application specific integrated circuits (ASICs), a plurality of suitably configured digital logic gates, and so forth.

The memory214may include one or a combination of volatile memory elements (e.g., random-access memory (RAM) such as DRAM and SRAM) and nonvolatile memory elements (e.g., ROM, hard drive, tape, CDROM). The memory214typically comprises a native operating system216, one or more native applications, emulation systems, or emulated applications for any of a variety of operating systems and/or emulated hardware platforms, emulated operating systems, etc. For example, the applications may include application specific software that may comprise some or all the components of the computing device102displayed inFIG.1.

In accordance with such embodiments, the components are stored in memory214and executed by the processing device202, thereby causing the processing device202to perform the operations/functions disclosed herein. For some embodiments, the components in the computing device102may be implemented by hardware and/or software.

Input/output interfaces204provide interfaces for the input and output of data. For example, where the computing device102comprises a personal computer, these components may interface with one or more input/output interfaces204, which may comprise a keyboard or a mouse, as shown inFIG.2. The display208may comprise a computer monitor, a plasma screen for a PC, a liquid crystal display (LCD) on a hand held device, a touchscreen, or other display device.

In the context of this disclosure, a non-transitory computer-readable medium stores programs for use by or in connection with an instruction execution system, apparatus, or device. More specific examples of a computer-readable medium may include by way of example and without limitation: a portable computer diskette, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM, EEPROM, or Flash memory), and a portable compact disc read-only memory (CDROM) (optical).

Reference is made toFIG.3, which is a flowchart300in accordance with various embodiments for adjusting the lighting intensity of a face chart to enhance a 3D effect, where the operations are performed by the computing device102ofFIG.1. It is understood that the flowchart300ofFIG.3provides merely an example of the different types of functional arrangements that may be employed to implement the operation of the various components of the computing device102. As an alternative, the flowchart300ofFIG.3may be viewed as depicting an example of steps of a method implemented in the computing device102according to one or more embodiments.

Although the flowchart300ofFIG.3shows a specific order of execution, it is understood that the order of execution may differ from that which is displayed. For example, the order of execution of two or more blocks may be scrambled relative to the order shown. In addition, two or more blocks shown in succession inFIG.3may be executed concurrently or with partial concurrence. It is understood that all such variations are within the scope of the present disclosure.

At block310, the computing device102obtains an image depicting a face of a user. At block320, the computing device102identifies facial features in the image and extracts characteristics of the facial features in the image. At block330, the computing device102generates a two-dimensional (2D) face chart based on the facial feature characteristics.

At block340, the computing device102predicts a skin tone of the user's face depicted in the image of the user. For some embodiments, the computing device102predicts the skin tone of the user's face depicted in the image of the user by applying a machine-learning algorithm. At block350, the computing device102changes color in a color map of a predefined 3D model based on the predicted skin tone.

At block360, the computing device102selects a predefined environment map based on characteristics in the image depicting the face of the user. For some embodiments, the characteristics comprise an average value of luminance, chrominance, chroma, color temperature, and/or contrast levels in the image. The environment map may comprise, for example, information relating to environmental lighting, predefined environmental lighting associated with different events, a high dynamic range image (HDRI) map, and so on. The events may include, for example, indoor events and outdoor events.

At block370, the computing device102generates a target face image based on the predefined 3D model. For some embodiments, the computing device102generates the target face image based on the predefined 3D model using physically based rendering and image-based lighting (IBL).

For some embodiments, the computing device102superimposes the target face image onto the 2D face chart based on locations of facial features in the target face image. For some embodiments, the computing device102generates a final 2D face chart based on the target face image by superimposing the target face image onto the 2D face chart based on locations of facial features in the target face image. Thereafter, the process inFIG.3ends.

To illustrate various features of the present invention, reference is made to the figures described below.FIG.4illustrates an example user interface402provided on a display of the computing device102whereby an image of the user's face404is captured and displayed to the user. For some implementations, the image capture module106(FIG.1) executing in the computing device102may be configured to cause a front-facing camera of the computing device102to capture an image or a video of a user's face404for purposes of generating a face chart of the user's face. The computing device102may also be equipped with the capability to connect to the Internet, and the image capture module106may be configured to obtain an image or video of the user from another device or server.

FIG.5illustrates the computing device102ofFIG.1generating a 2D face chart from an image of the user's face according to various embodiments. As described above, the image capture module106obtains a digital image502of a user's facial region and displays the user's face. The 2D face chart generator108identifies facial features depicted in the image502obtained by the image capture module106and extracts characteristics of the facial features in the image. Such characteristics may comprise the size and shape of each facial feature depicted in the image, the overall size and shape of the user's face, contouring of the user's face, and so on. The 2D face chart generator108generates a 2D face chart504based on these facial feature characteristics, where the 2D face chart504depicts the user's facial features. The 2D face chart generator108is further configured to predict the skin tone of the user's face depicted in the image and adjust the color of the 2D face chart504according to the predicted skin tone. For some embodiments, the 2D face chart generator108predicts the skin tone of the user's face depicted in the image of the user is performed by applying a machine-learning algorithm.

FIG.6illustrates the computing device102ofFIG.1generating a final 2D face chart604and adjusting lighting effects on the final 2D face chart604according to various embodiments. The lighting analyzer110receives the image502of the user's face obtained by the image capture module106and analyzes the lighting environment depicted in the image502to obtain lighting characteristics of the user's face. Such lighting characteristics may include the lighting intensity of each pixel in the facial region of the user, which may be attributed to, for example, positioning of one or more light sources with respect to the user's face. For some embodiments, the lighting analyzer110is configured to select a predefined environment map based on, for example, an average value of luminance, chrominance, chroma, color temperature, and/or contrast levels in the image. The environment map may comprise, for example, information relating to environmental lighting, predefined environmental lighting associated with different events, a high dynamic range image (HDRI) map, and so on. The events may include, for example, indoor events and outdoor events.

The 3D model retriever112in the 3D model module113analyzes the facial characteristics obtained by the 2D face chart generator108(FIG.5). Such facial characteristics may include the size and shape of each facial feature depicted in the image, the overall size and shape of the user's face, contouring of the user's face, and so on. For some embodiments, the 3D model retriever112obtains lighting characteristics of the user's face depicted in the image by obtaining a luminance, chrominance, chroma, color temperature, or contrast level for each pixel in the user's face depicted in the image502and determining an average value of the luminance, chrominance, chroma, color temperature, or contrast levels of the pixels in the user's face.

The 3D model retriever112accesses a data store118containing predefined 3D models120and retrieves a 3D model120that most closely matches the user's face based on the facial characteristics. The lighting adjuster114generates one or more lighting effects on the final 2D face chart604based on the lighting characteristics of the user's face depicted in the image, thereby enhancing a stereoscopic effect of the final 2D face chart604. For some embodiments, the lighting adjuster114generates the one or more lighting effects on the final 2D face chart604by obtaining a luminance, chrominance, chroma, color temperature, or contrast level for each pixel in the image, determining an average value of the luminance, chrominance, chroma, color temperature, or contrast levels of the pixels in the image for selecting a predefined environment map.

The lighting adjuster114generates the one or more lighting effects by determining a new brightness level for each pixel in the 3D face chart604effects based on a weighting value, the average value of the brightness levels of the pixels in the user's face, and the average value of the brightness levels of the pixels in the 2D face chart.

For some embodiments, the lighting adjuster114determines the new brightness level for each pixel in the 3D face chart604based on a ratio of the average value of all the brightness levels of the pixels in the user's face to the average value of all the brightness levels of the pixels in the 2D face chart. In particular, let the value (T) represent the brightness level for a particular pixel in the image502of the user. The lighting adjuster114obtains an average value of the brightness levels from the image502of the user's face (represented by X). The lighting adjuster114similarly obtains an average value of the brightness levels from the 3D face chart604(represented by Y). The light adjuster114calculates a new brightness level (T′) for a corresponding pixel in the 3D chart based on the following expression:

A lighting effect based on the new brightness level (T′) is applied by the light adjuster114to each pixel in the 3D face chart604. The light effect applied to the 3D face chart604simulates the lighting environment (e.g., the shading) depicted in the image502of the user's face, thereby enhancing a stereoscopic effect of the 3D face chart604.