Patent Description:
A wide variety skin care products are marketed for treating skin conditions, but it is not uncommon for a consumer to have difficulty determining which skin care product she should use. As a result, consumers may purchase a skin care product that does not treat the particular skin condition for which the consumer is seeking treatment. Accordingly, there remains a need for skin care products that are customized for a consumer's needs.

<CIT> ("the '<NUM> Publication") relates to a method for assessment of aesthetic and morphological conditions of the skin and prescription of cosmetic and/or dermatological treatment. The '<NUM> Publication describes obtaining information on the age and life habits of a user; performing a biometric analysis of a corporeal portion of the user; processing the results from the analyses and comparing them with predetermined data on aesthetic and morphological factors of the skin; obtaining a skin model of the user according to the data processed; linking the user's skin model to at least two-dimensional predetermined data contained in a database about cosmetic and/or dermatological products; and prescribing a kit of pre-selected cosmetic and/or dermatological products. While the '<NUM> Publication describes performing an analysis of a user's skin and performing treatment based on the analysis, the '<NUM> Publication fails to utilize a convolutional neural network. The '<NUM> Publication also fails to describe evaluating skin features and performing a comparison with a baseline.

<CIT> ("the '<NUM> Patent") relates to a system and method of cosmetic analysis and treatment diagnosis. The '<NUM> Patent describes receiving assessment data of observable characteristics of each of a plurality of defined body areas of a subject; converting the assessment data for each of the plurality of defined body areas to weighted data associated with each body area; generating cosmetic analysis data from the weighted data; and outputting the cosmetic analysis data. Thus, while the '<NUM> Patent describes systems that may perform a cosmetic analysis, the '<NUM> Patent and other prior art fails to disclose evaluating skin preferences or determining product preferences.

<CIT>relates to a system and a method to analyse the apparent age of a person's skin and suggest treatments.

Accordingly, there is a need for an improved method of evaluating consumer skin conditions and providing a customized product recommendation based on said evaluation.

Included are embodiments for providing customized product recommendations. Some embodiments include a system that includes an image capture device and a computing device that is coupled to the image capture device. The computing device may include a memory component that stores logic that causes the system to capture an image of a user via the image capture device and process the image through a convolutional neural network to determine a skin age of the user. Determining the skin age may include identifying at least one pixel that is indicative of the skin age and utilizing the at least one pixel to create a heat map that identifies a region of the image that contributes to the skin age. The logic may further cause the system to determine a target skin age of the user, determine a product for the user to apply to a region of skin of the user for achieving the target skin age, and provide an option for the user to purchase the product.

Also included are embodiments of a method. Some embodiments of the method include receiving an image of a user and processing the image through a convolutional neural network to determine a skin age of the user. Determining the skin age may include identifying at least one pixel that is indicative of the skin age and utilizing the pixel to create a two-dimensional heat map that identifies a region of the image that contributes to the skin age. The method may also include determining a product for the user to apply to a region of skin for achieving a target skin age, where the product is determined as being applicable to the region and providing an option for the user to purchase the product.

Also included are embodiments of a non-transitory computer-readable medium. Some embodiments of the non-transitory computer-readable medium include logic that causes a computing device to receive an image of a user and create a two-dimensional heat map of the image, where the two-dimensional heat map is created via a convolutional neural network to identify at least one pixel of the image that is indicative of a skin age, and where the two-dimensional heat map identifies a region of the image that contributes to the skin age. The logic may further cause the computing device to determine, from the two-dimensional heat map, a target skin age of the user and determine a product for the user to apply to a region of skin for achieving the target skin age. In some embodiments, the logic causes the computing device to provide an option for the user to purchase the product.

The drawings illustrate various embodiments described herein, and together with the description serve to explain the principles and operations of the claimed subject matter.

A variety of systems and methods have been used in the cosmetics industry to provide customized product recommendations to consumers. For example, systems that use a feature-based analysis, in which one or more features of a skin condition (e.g., fine lines, wrinkles, spots, uneven skin tone) are detected in a captured image (e.g., digital photo) by looking for features that meet a predefined definition, are commonly used for analyzing skin. However, feature based analysis systems rely on predetermined definitions for the particular skin conditions of interest and can require substantial computer memory and/or processing power. As a result, feature-based systems may not provide the desired level of accuracy when diagnosing a skin condition or determining skin age.

In view of the drawbacks of some conventional feature-based image analysis systems, the methods and systems described herein rely on a convolutional neural network ("CNN") based system to provide a user with an analysis of skin age and indications of skin conditions. The CNN based image analysis system herein uses relatively little image pre-processing, which reduces the dependence of the system on prior knowledge and predetermined definitions. Consequently, the present system demonstrates improved generalization compared to a conventional feature-based image analysis system and can provide a more accurate skin analysis and/or age prediction, which may lead to a better skin care product or regimen recommendations for a consumer who uses the system.

"Anchor feature" means a feature on the face of a user that is utilized for normalizing an image of the user's face.

"Convolutional neural network" is a type of feed-forward artificial neural network where the individual neurons are tiled in such a way that they respond to overlapping regions in the visual field.

"Data augmentation" means altering data associated with a training image or other image to create additional samples for the image.

"Heat map" herein refers to a digital image of a user's face in which portions of the image are visually highlighted to identify skin features and/or areas (e.g., forehead, cheek, nasolabial folds, crow's feet, under eye, upper lip) that contributes to a determined skin age.

"Image capture device" means a device such as a digital camera capable of capturing an image of a user;.

"Skin age" means the age of a user's skin calculated by the system herein, based on a captured image.

"Target skin age" means a skin age that is a predetermined number of years less than the skin age.

"User" herein refers to any person who uses at least the features provided herein, including, for example, a device user, a product user, a system user, and the like.

The systems and methods herein use a trained convolutional neural network, which functions as an in silico skin model, to predict the skin age of a user by analyzing a captured image of the skin of the user (e.g., facial skin). The CNN comprises multiple layers of neuron collections that use the same filters for each pixel in a layer. Using the same filters for each pixel in the various combinations of partially and fully connected layers reduces memory and processing requirements of the system.

In some instances, the system may include a preprocessing stage followed by a stage for CNN training and image analysis. During preprocessing, one or more facial features common to most users, such as eyes, forehead, cheeks, nose, under eye region, outer eye region, nasolabial folds, lips, and portions of the face adjacent these features ("anchor features"), in a received image may be detected. The system may detect the anchor feature(s) using known edge detection techniques, shape detection techniques, and the like. Based on the location of the anchor feature(s), the image may be scaled and rotated to make the image substantially level and with the anchor feature(s) arranged in a predetermined position in the final image. In this way, training images can be consistently aligned, thus providing more consistent training and analysis. The image may then be cropped to a predetermined area of pixels as input for further processing.

Preprocessing may also include image normalization. For example, global contrast normalization may be utilized to standardize the training images (and/or images of users). Similarly, the images may be masked with a fixed (or predetermined) size oval mask to minimize the influence of other features like hair, neck and other undesired objects in the image.

In some instances, data augmentation may be performed to create additional samples from an inputted image. The additional samples are used to train the CNN to tolerate variation in input images. This helps improve the accuracy of the model. In other words, the CNN is able to extract the information necessary for a suitable analysis in spite of differences in, for example, the way people take photographs, the conditions in which photos are taken, and the hardware used to take a photo. The additional samples generated by data augmentation can also force the CNN to learn to rely on a variety of features for skin age prediction, rather than one particular feature, and may prevent over-training of the CNN. Some non-limiting examples of data augmentation include randomly enlarging or shrinking the image, randomly rotating the image in a clockwise or counterclockwise direction, randomly cropping the image, and/or randomly changing the saturation and/or exposure of the image. In some instances the image data may be augmented by subjecting the input image to random vertical dropout, in which a random column of pixels is removed from the image.

The CNN herein may be trained using a deep learning technique, which allows the CNN to learn what portions of an image contribute to skin age, much in the same way as a mammalian visual cortex learns to recognize important features in an image. In some instances, the CNN training may involve using mini-batch stochastic gradient descent (SGD) with Nesterov momentum (and/or other algorithms). An example of utilizing a stochastic gradient descent is disclosed in <CIT>.

In some instances, the CNN may be trained by providing an untrained CNN with a multitude of captured images to learn from. In some instances, the CNN can learn to identify portions of skin in an image that contribute to skin age through a process called supervised learning. "Supervised learning" generally means that the CNN is trained by analyzing images in which the age of the person in the image is predetermined. Depending on the accuracy desired, the number of training images may vary from a few images to a multitude of images (e.g., hundreds or even thousands) to a continuous input of images (i.e., to provide continuous training).

The systems and methods herein utilize a trained CNN that is capable of accurately predicting the apparent age of a user for a wide range of skin types. To generate a predicted age, an image of a user (e.g., digital image of a user's face) is forward-propagating through the trained CNN. The CNN analyzes the image and identifies portions of skin in the image that contribute to the predicted age of the user ("trouble spots"). The CNN then uses the trouble spots to predict the age of the user. The system then determines a target skin age (e.g., the predicted age of the user minus a predetermined number of years (e.g., <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, or <NUM> year(s) or the actual age of the user), and a gradient is propagated back to the original image. The absolute value of a plurality of channels of the gradient may then be summed for at least one pixel and scaled from <NUM>-<NUM> for visualization purposes. The value of the scaled pixels may represent pixels that contribute most (and least) to the determination of the skin age of the user. Each scaling value (or range of values) may be assigned a color or shade, such that a virtual mask can be generated to graphically represent the scaled values of the pixels. These pixels are then arranged to form part of a two-dimensional heat map that indicates the areas on the user's face that drive the skin age (perceived age) of the consumer. In some instances, the CNN analysis and/or target age, optionally in conjunction with habits and practices input provided by a user, can be used to help provide a skin care product and/or regimen recommendation.

<FIG> depicts a system <NUM> for capturing an image of a user, analyzing the image, and providing a customized product recommendation. The system <NUM> may include a network <NUM>, which may be embodied as a wide area network (such as a mobile telephone network, a public switched telephone network, a satellite network, the internet, etc.), a local area network (such as wireless-fidelity, Wi-Max, ZigBee™, Bluetooth™, etc.), and/or other forms of networking capabilities. Coupled to the network <NUM> are a mobile computing device <NUM>, a remote computing device <NUM>, a kiosk computing device <NUM>, and a training computing device <NUM>.

The mobile computing device <NUM> may be a mobile telephone, a tablet, a laptop, a personal digital assistant and/or other computing device configured for capturing, storing, and/or transferring an image such as a digital photograph. Accordingly, the mobile computing device <NUM> may include an image capture device <NUM> such as a digital camera and/or may be configured to receive images from other devices. The mobile computing device <NUM> may include a memory component 140a, which stores image capture logic 144a and interface logic 144b. The memory component 140a may include random access memory (such as SRAM, DRAM, etc.), read only memory (ROM), registers, and/or other forms of computing storage hardware. The image capture logic 144a and the interface logic 144b may include software components, hardware circuitry, firmware, and/or other computing infrastructure, as described herein. As described in more detail below, the image capture logic 144a may facilitate capturing, storing, preprocessing, analyzing, transferring, and/or performing other functions on a digital image of a user. The interface logic 144b may be configured for providing one or more user interfaces to the user, which may include questions, options, and the like. The mobile computing device <NUM> may also be configured for communicating with other computing devices via the network <NUM>.

The remote computing device <NUM> may also be coupled to the network <NUM> and may be configured as a server (or plurality of servers), personal computer, mobile computer, and/or other computing device configured for creating and training a convolutional neural network capable of determining the skin age of a user by identifying portions of skin in a captured image that contribute to skin age. Commonly perceived skin flaws such as fine lines, wrinkles, dark (age) spots, uneven skin tone, blotchiness, enlarged pores, redness, yellowness, combinations of these and the like may all be identified by the trained CNN as contributing to the skin age of the user. The remote computing device <NUM> may include a memory component 140b, which stores training logic 144c and analyzing logic 144d. The training logic 144c may facilitate creation and/or training of the CNN, and thus may facilitate creation of and/or operation of the CNN. For example, the CNN may be stored as logic 144c, 144d in the memory component 140b of a remote computing device <NUM>. The analyzing logic 144d may cause the remote computing device <NUM> to receive data from the mobile computing device <NUM> (or other computing device) and process the received data for providing a skin age, product recommendation, etc..

The system <NUM> may also include a kiosk computing device <NUM>, as illustrated in <FIG>. The kiosk computing device <NUM> may operate similar to the mobile computing device <NUM>, but may also be able to dispense one or more products and/or receive payment in the form of cash or electronic transactions. In some instances, the kiosk computing device <NUM> may also be configured to facilitate training of the CNN, as described in more detail below with regard to the training computing device <NUM>.

A training computing device <NUM> may be coupled to the network <NUM> to facilitate training of the CNN. For example, a trainer may provide one or more digital images of a face or skin to the CNN via the training computing device <NUM>. The trainer may also provide information and other instructions to inform the CNN which assessments are correct and which assessments are not correct. Based on the input from the trainer, the CNN may automatically adapt, as described in more detail below.

It should be understood that while the kiosk computing device <NUM> is depicted as a vending machine type of device, this is merely an example. Some embodiments may utilize a mobile device that also provides payment and/or production dispensing. Similarly, the kiosk computing device <NUM>, the mobile computing device <NUM>, and/or the training computing device <NUM> may be utilized for training the CNN. As a consequence, the hardware and software depicted for the mobile computing device <NUM> and the remote computing device <NUM> may be included in the kiosk computing device <NUM>, the training computing device <NUM>, and/or other devices. Similarly, the hardware and software depicted for the remote computing device <NUM> in <FIG> may be included in one or more of the mobile computing device <NUM>, the remote computing device <NUM>, the kiosk computing device <NUM>, and the training computing device <NUM>.

It should also be understood that while the remote computing device <NUM> is depicted in <FIG> as performing the convolutional neural network processing, this is merely an example. The convolutional neural network processing may be performed by any suitable computing device, as desired.

<FIG> depicts an example of a convolutional neural network <NUM> for use in the present system. The CNN <NUM> may include an inputted image <NUM>, one or more convolution layers C<NUM>, C<NUM>, one or more subsampling layers S<NUM> and S<NUM>, one or more partially connected layers, one or more fully connected layers, and an output. To begin an analysis or to train the CNN, an image <NUM> is inputted into the CNN <NUM> (e.g., the image of a user). The CNN may sample one or more portions of the image to create one or more feature maps in a first convolution layer C<NUM>. For example, as illustrated in <FIG>, the CNN may sample six portions of the image <NUM> to create six features maps in the first convolution layer C<NUM>. Next, the CNN may subsample one or more portions of the feature map(s) in the first convolution layer C<NUM> to create a first subsampling layer S<NUM>. In some instances, the subsampled portion of the feature map may be half the area of the feature map. For example, if a feature map comprises a sample area of <NUM> x <NUM> pixels from the image <NUM>, the subsampled area may be <NUM> x <NUM> pixels. The CNN <NUM> may perform one or more additional levels of sampling and subsampling to provide a second convolution layer C<NUM> and a second subsampling layer S<NUM>. It is to be appreciated that the CNN <NUM> may include any number of convolution layers and subsampling layers as desired. Upon completion of final subsampling layer (e.g., layer S<NUM> in <FIG>), the CNN <NUM> generates a fully connected layer F<NUM>, in which every neuron is connected to every other neuron. From the fully connected layer F<NUM>, the CNN can generate an output such as a predicted age or a heat map.

<FIG> depict a system and method for analyzing a captured image <NUM>. The captured image <NUM> may be captured by a user, for example, using the image capture device <NUM> illustrated in <FIG>. <FIG> illustrates a first action taken in preprocessing, the identification of anchor features 332a - 332e. The anchor features 332a - 332e illustrated in <FIG> include eyes 332a and 332c, nose or nostril(s) 332d, and corners of the mouth 332b and 332e. But it is to be appreciated that any prominent or detectable facial feature(s) may be an anchor feature. While the anchor features 332a - 332e are visually depicted in <FIG>, this is merely an example. Some embodiments do not provide a visual representation of anchor features 332a - 332e on the mobile computing device <NUM>.

Once the anchor features 332a - 332e are identified, the captured image <NUM> may be normalized such that the anchor features 332a - 332e are arranged in a predetermined position. In some instances, normalization may include rotating the image, resizing the image, and/or performing other image adjustments. In <FIG>, the captured image <NUM> may be further preprocessed by cropping to remove one or more unwanted portions of the captured image <NUM>, thereby creating a normalized image <NUM>. As an example, background, hair, ears, and/or other portions of the captured image <NUM> may be removed.

Once the image <NUM> has been preprocessed, an analysis of the normalized image <NUM> may be performed. As described above, the CNN <NUM> may be configured to determine locations, colors, and/or shade (i.e., lightness or darkness) of pixels that contribute to a skin age of the user. Depending on the particular CNN, each skin age may have different characteristics. Accordingly, the mobile computing device <NUM>, the remote computing device <NUM>, and/or the kiosk computing device <NUM> may segment the image into a plurality of regions that are typically utilized in determining skin age. <FIG> illustrates a first layer of a heat map, which may be described as a mask of interest <NUM>. The mask of interest <NUM> identifies a plurality of regions 338a - 338i that have been identified by the CNN <NUM> as contributing to the age of the user. For example, the mask may include a forehead region (338a), one or more under eye regions 338d and 338e, one or more outer eye regions 338b and 338c, one or more cheek regions 338f and <NUM>, an upper lip region 338j, and one or more nasolabial fold regions <NUM> and 338i. The determination of the mask of interest <NUM> and/or the plurality of regions 338a - 338i may be customized based on the shape and/or size of the user's face, such that the regions more accurately reflect the areas that affect age determination for each user.

<FIG> depicts a second layer of a heat map, which may also be described as a pixel map <NUM>. The pixel map <NUM> includes a multitude of scaled pixels in portions of the image identified by the system as contributing to the age of the user. As described above, a pixel-by-pixel examination of the input image may be made regarding identifiers of age to identify at least one pixel that contributes to the skin age determination. The location, shade, and/or color of each pixel may be determined and mapped, as illustrated in <FIG>, where the lighter pixels are identified as being a higher indicator of skin age than the darker pixels. In <FIG>, the pixel map <NUM> and the mask of interest <NUM> may overlay the normalized image <NUM> to create a two-dimensional heat map <NUM>. The two-dimensional heat map <NUM> may indicate regions of skin that the CNN <NUM> identifies as contributing to an elevated age.

Once the CNN identifies the areas that are contributing to the predicted age of the user and generates a predicted age of the user, a two-dimensional heat map <NUM> can be created and the regions of interest displayed to the user for example on a mobile computing device. In some instances, a predetermined time period may be subtracted from the predicted skin age to provide a target skin age. The predicted age and/or target age may be displayed to the user, for example, on a mobile computing device. The target skin age and the regions of interest may then be utilized by the system to determine a beauty regimen and/or product for the user to utilize to realize the target skin age. In some embodiments, recommendations may be made to maintain a skin age. As an example, a user's actual age may be young enough that maintenance might be the goal. Similarly, if a user's age is determined by the CNN to be equal to or less than the user's actual age, maintenance products and/or regimens may be recommended. Additional images may be taken during or after use of a recommended product and/or regimen to monitor progress and/or revise the regimen or product recommendation.

In some instances, at least some of the images and other data described herein may be stored as historical data for later use. As an example, tracking of user progress may be determined based on this historical data. Other analyses may also be performed on this historical data, depending on the embodiment.

<FIG> depicts a user interface <NUM> for capturing an image of a user and for providing customized product recommendations. As illustrated, the mobile computing device <NUM> may provide an application for capturing an image of a user. Accordingly, <FIG> depicts an introductory page on the mobile computing device <NUM> for beginning the process of capturing an image and providing customized product recommendations. The user interface <NUM> also includes a start option <NUM> for beginning the process.

<FIG> depicts a user interface <NUM> illustrating an image that is analyzed for providing customized product recommendations, according to embodiments described herein. In response to selection of the start option <NUM> from <FIG>, the user interface <NUM> may be provided. As illustrated, the image capture device <NUM> may be utilized for capturing an image of the user. In some embodiments, the user may utilize a previously captured image. Regardless, upon capturing the image, the image may be provided in the user interface <NUM>. If the user does not wish the image be utilized, the user may retake the image. If the user approves the image, the user may select the next option <NUM> to begin analyzing the image and proceeding to the next user interface.

<FIG> depicts a user interface <NUM> providing a normalized image <NUM> and a pixel map <NUM> that may be created for product recommendations. In response to selection of the next option <NUM> from <FIG>, the user interface <NUM> may be provided, which may present the user with an age-input option <NUM>. An additional-predictions option <NUM> may also be provided.

<FIG> depicts a user interface <NUM> for providing a questionnaire to a user to help customize product recommendations. In response to entering a real age in the age-input option <NUM> of <FIG>, the user interface <NUM> may be provided. As illustrated, the user interface <NUM> may provide one or more questions for determining additional details regarding the user, including product preferences, current regimens, etc. As an example, the questions may include whether the user utilizes a moisturizer with sunscreen. One or more predefined answers <NUM> may be provided for the user to select from.

<FIG> depicts a user interface <NUM> for providing additional prompts for a questionnaire, according to embodiments described herein. In response to entering the requested data from the user interface <NUM> of <FIG>, the user interface <NUM> may be provided. As illustrated, the user interface <NUM> provides another question (such as whether the user prefers scented skin care products) along with three predefined answers <NUM> for the user to select from. A submit option <NUM> may also be provided for submitting the selected answer(s).

It should be understood that while <FIG> and <FIG> provide two questions, any number of questions may be provided to the user, depending on the particular embodiment. The questions and number of questions may depend on the user's actual age, on the user's skin age, and/or other factors.

<FIG> depicts a user interface <NUM> for providing a skin age of a user, based on a captured image, according to embodiments described herein. In response to completing the questionnaire of <FIG> and <FIG>, the user interface <NUM> may be provided. As illustrated, the user interface <NUM> may provide the user's skin age and the captured image with at least one identifier <NUM> to indicate which region(s) of the user's skin are contributing to the user age predicted by the CNN. In some instances, the system may also provide a list <NUM> of the areas that contribute to the user's predicted age. A description <NUM> may also be provided, as well as a product-recommendation option <NUM> for viewing customized product recommendations.

<FIG> depicts a user interface <NUM> for providing product recommendations, according to embodiments described herein. In response to selection of the product-recommendation option <NUM> from <FIG>, the user interface <NUM> may be provided. As illustrated, the user interface <NUM> may provide one or more recommended products that were determined based on the user's age, areas contributing to the user's age and the target age. Specifically, the at least one product may be determined as being applicable to the region of skin of the user that contributes to the predicted age of the user. As an example, creams, moisturizers, lotions, sunscreens, cleansers and the like may be recommended. Also provided is a regimen option <NUM> for providing a recommended regimen. A purchase option <NUM> may also be provided.

<FIG> depicts a user interface <NUM> for providing details of product recommendations, according to embodiments described herein. In response to selection of the regimen option <NUM> from <FIG>, the user interface <NUM> may be provided. As illustrated, the user interface <NUM> may provide a products option <NUM> and a schedule option <NUM> for using the recommended product in the user's beauty regimen. Additional information related to the first stage of the beauty regimen may be provided in section <NUM>. Similarly, data related to a second and/or subsequent stage of the regimen may be provided in the section <NUM>.

<FIG> depicts a user interface <NUM> that provides a recommended beauty regimen. In response to selection of the schedule option <NUM> from <FIG>, the user interface <NUM> may be provided. The user interface <NUM> may provide a listing of recommended products, as well as a schedule, including schedule details for the regimen. Specifically, the user interface <NUM> may provide a time of day that products may be provided. A details option <NUM> may provide the user with additional details regarding products and the regimen.

<FIG> depicts a user interface <NUM> for providing additional details associated with a beauty regimen and the products used therein. The user interface <NUM> may be provided in response to selection of the details option <NUM> from <FIG>. As illustrated, the user interface <NUM> may provide details regarding products, application tips, etc. Additionally, a "science-behind" option <NUM>, <NUM> and a "how-to-demo" option <NUM>, <NUM> may be provided. In response to selection of the science behind option <NUM>, <NUM>, details regarding the recommended product and the application regimen may be provided. In response to selection of the how to demo option <NUM>, <NUM>, audio and/or video may be provided for instructing the user on a strategy for applying the product. Similarly, the subsequent portions of the regimen (such as step <NUM> depicted in <FIG>) may also include a science behind option <NUM>, <NUM> and a how to demo option <NUM>, <NUM>.

<FIG> depicts a user interface <NUM> for providing recommendations related to a determined regimen, according to embodiments described herein. In response to selection of the purchase option <NUM> (<FIG>), the user interface <NUM> may be provided. As illustrated, the user interface <NUM> includes purchasing options <NUM>, <NUM>, <NUM> for purchasing one or more recommended products. The user interface <NUM> may also provide an add-to-cart option <NUM> and a shop-more option <NUM>.

<FIG> depicts a user interface <NUM> for providing product recommendations to a user timeline, according to embodiments described herein. As illustrated, the user interface <NUM> may provide a notification that one or more of the recommended products have been added to the user's timeline. Upon purchasing a product (e.g., via the user interface <NUM> from <FIG>), the purchased products may be added to the recommended regimen for the user. As such, the notification may include an acceptance option <NUM> and a view timeline option <NUM>.

<FIG> depicts a flowchart for training a convolutional neural network for identifying a feature from an image, according to embodiments described herein. As illustrated in block <NUM>, the CNN may be trained. As described above, the CNN may be trained utilizing training images and a convolutional neural network. In block <NUM>, data augmentation may be performed. Specifically, to increase the robustness of the CNN, data augmentation techniques may be utilized to create additional samples from the training images. As described above, some embodiments may be configured to randomly zoom in and zoom out of the image; perform a random rotation of the image in a clockwise direction and/or in a counter clockwise direction; randomly crop the image; randomly change the saturation and exposure of the input image; utilize vertical dropout to randomly dropout a column of pixels (feature map) of an image; etc. In block <NUM>, one or more of the training images may be normalized. Normalization may include cropping, rotation, zooming, removal of background imagery, etc. In block <NUM>, masking may be performed. Masking may include identifying areas of interest to determine skin age, as well as creating a heat map for representing the areas of interest. In block <NUM>, the CNN architecture may be created via the convolutional neural network.

<FIG> depicts a flowchart for generating a heat map of an image, which may be utilized for feature recognition, according to embodiments described herein. As illustrated in block <NUM>, an image may be processed. As described above, an image may be processed, such as by cropping, rotating, zooming, etc. In block <NUM>, the image may be propagated through the CNN. As described above, the image may be processed through the convolutional neural network to identify pixels, regions, and/or areas that signify a skin age. In block <NUM>, the skin age may be predicted. In block <NUM>, a heat map may be generated.

<FIG> depicts a flowchart for providing customized product recommendations, according to embodiments described herein. In block <NUM>, an image of a user may be captured. In block <NUM>, questions may be provided to the user. In block <NUM>, answers to the questions may be received from the user. In block <NUM>, a skin age may be provided to the user. In block <NUM>, a heat map may be generated based on a convolutional neural network. In block <NUM>, a customized product recommendation may be provided to the user.

<FIG> depicts components of a remote computing device <NUM> for providing customized product recommendations, according to embodiments described herein. The remote computing device <NUM> includes a processor <NUM>, input/output hardware <NUM>, network interface hardware <NUM>, a data storage component <NUM> (which stores image data 1938a, product data 1938b, and/or other data), and the memory component 1940b. The memory component 1940b may be configured as volatile and/or nonvolatile memory and as such, may include random access memory (including SRAM, DRAM, and/or other types of RAM), flash memory, secure digital (SD) memory, registers, compact discs (CD), digital versatile discs (DVD), and/or other types of non-transitory computer-readable mediums. Depending on the particular embodiment, these non-transitory computer-readable mediums may reside within the remote computing device <NUM> and/or external to the remote computing device <NUM>.

The memory component 1940b may store operating logic <NUM>, the training logic 1944c and the analyzing logic 1944d. The training logic 1944c and the analyzing logic 1944d may each include a plurality of different pieces of logic, each of which may be embodied as a computer program, firmware, and/or hardware, as an example. A local communications interface <NUM> is also included in <FIG> and may be implemented as a bus or other communication interface to facilitate communication among the components of the remote computing device <NUM>.

The processor <NUM> may include any processing component operable to receive and execute instructions (such as from a data storage component <NUM> and/or the memory component 1940b). As described above, the input/output hardware <NUM> may include and/or be configured to interface with the components of <FIG>.

The network interface hardware <NUM> may include and/or be configured for communicating with any wired or wireless networking hardware, including an antenna, a modem, a LAN port, wireless fidelity (Wi-Fi) card, WiMax card, Bluetooth™ module, mobile communications hardware, and/or other hardware for communicating with other networks and/or devices. From this connection, communication may be facilitated between the remote computing device <NUM> and other computing devices, such as those depicted in <FIG>.

The operating system logic <NUM> may include an operating system and/or other software for managing components of the remote computing device <NUM>. As discussed above, the training logic 1944c may reside in the memory component 1940b and may be configured to cause the processor <NUM> to train the convolutional neural network. Similarly, the analyzing logic 1944d may be utilized to analyze images for skin age prediction.

It should be understood that while the components in <FIG> are illustrated as residing within the remote computing device <NUM>, this is merely an example. In some embodiments, one or more of the components may reside external to the remote computing device <NUM> and/or the remote computing device <NUM> may be configured as a mobile device. It should also be understood that, while the remote computing device <NUM> is illustrated as a single device, this is also merely an example. In some embodiments, the training logic 1944c and the analyzing logic 1944d may reside on different computing devices. As an example, one or more of the functionalities and/or components described herein may be provided by the mobile computing device <NUM> and/or other devices, which may be communicatively coupled to the remote computing device <NUM>. These computing devices may also include hardware and/or software for performing the functionality described herein.

Additionally, while the remote computing device <NUM> is illustrated with the training logic 1944c and the analyzing logic 1944d as separate logical components, this is also an example. In some embodiments, a single piece of logic may cause the remote computing device <NUM> to provide the described functionality.

Claim 1:
A system for providing customized product recommendations to a user based on a skin age analysis, comprising:
a) an image capture device;
b) an image of a user captured by the image capture device; and
c) a computing device coupled to the image capture device, wherein the computing device includes a memory component that stores logic that causes the system to
(i) analyze the captured image of the user using a trained convolutional neural network to predict a user's skin age, wherein the trained convolutional neural network comprises multiple layers of neuron collections that use the same filters for each pixel in a layer and wherein predicting the user's skin age includes identifying at least one pixel in the captured image that is indicative of the user's skin age, where the at least one pixel is identified by a pixel-by-pixel examination of the captured image regarding identifiers of age,
(ii) utilize the at least one pixel to generate a heat map that identifies a region of the image that contributes to the user's predicted skin age, where the heat map is a digital image of the user's face in which portions of the image are visually highlighted to identify skin features and/or areas that contribute to the predicted skin age,
(iii) display the heat map to the user on a display device visible to the user, and
(iv) recommend , based on the skin age analysis, a product for the user to apply to a region of skin for achieving a target skin age, the target skin age being one of: a predetermined number of years less than the skin age; or the actual age of the user.