Promotion processing system for generating a digital promotion based upon a satisfaction level determined from a facial image and related methods

A promotion processing system may include a remote device associated with a given user and that includes a camera. The promotion processing system may also include a promotion processing server configured to store meal recipes and associated ingredients, and obtain a selected meal recipe from the given user. The promotion processing server may also be configured to cooperate with the camera of the remote device to obtain a facial image of the given user during consumption of ingredients of the selected meal recipe, determine a satisfaction level with the selected meal recipe based upon the facial image, and generate a digital promotion toward another meal recipe having at least one common ingredient with the ingredients of the selected meal recipe based upon the satisfaction level. The promotion processing server may also communicate the digital promotion to the remote device.

TECHNICAL FIELD

The present application relates to the field of electronics, and, more particularly, to digital promotion systems, and related methods.

BACKGROUND

Sales of a particular product or service may be based upon how well that product or service is marketed to a consumer. One form of marketing is a coupon, typically in paper form, for a discount toward the product or service. Some coupons may be retailer-specific, for example, only redeemable for the discount at a particular retailer, while other coupons may be product-specific from a manufacturer and redeemable at any retailer.

A coupon, while typically in paper form, may be in digital form and may be referred to as a digital promotion. A digital promotion may be selected or “clipped” via a mobile phone and saved to a digital wallet for redemption at a point-of-sale (POS) terminal, for example. A typical coupon is applicable to a given product and has a redeemable value that may vary based upon the quantity of a given item, for example, brand of item, size of the product in terms of packaging, and/or the price point of the given item. A typical coupon may also be redeemable only at a given retailer and/or within a threshold time period.

A food recipe typically includes a set of instructions that describes how to prepare or make a meal or prepared food. Recipes were typically printed in books or other print media. With the increasing popularity of electronic commerce (e-commerce), an increasing amount of recipes are being published online.

SUMMARY

A promotion processing system may include a remote device associated with a given user and including a camera, and a promotion processing server. The promotion processing server may be configured to store a plurality of meal recipes and associated ingredients, obtain a selected meal recipe from the given user, and cooperate with the camera of the remote device to obtain a facial image of the given user during consumption of ingredients of the selected meal recipe. The promotion processing server may also be configured to determine a satisfaction level with the selected meal recipe based upon the facial image, generate a digital promotion toward another meal recipe having at least one common ingredient with the ingredients of the selected meal recipe based upon the satisfaction level, and communicate the digital promotion to the remote device.

The promotion processing server may be configured to apply image processing to the facial image to extract a plurality of facial features, and determine a facial expression of the given user during consumption of the ingredients based upon the plurality of facial features. The promotion processing server may also be configured to determine the satisfaction level based upon the facial expression, for example.

At least one of the plurality of facial features may include a visibility of teeth. At least one of the plurality of facial features may include a mouth-opening distance, for example. At least one of the plurality of facial features may include an eyebrow raise distance, for example.

The at least one digital promotion may have a redemption value associated therewith. The promotion processing server may be configured to set the redemption value based upon the satisfaction level, for example.

The promotion processing server may be configured to determine the another meal recipe based upon the satisfaction level. The promotion processing server may be configured to communicate the another meal recipe to the remote device, for example.

The digital promotion may be redeemable toward the common ingredient of the another meal recipe, for example. The promotion processing server may be configured to obtain the selected meal recipe based upon user input at the remote device. The remote device may include a mobile wireless communications device, for example.

A method aspect is directed to a method of processing a promotion. The method may include using a promotion processing server to store a plurality of meal recipes and associated ingredients, obtain a selected meal recipe from a given user, and cooperate with a camera of a remote device associated with the given user to obtain a facial image of the given user during consumption of ingredients of the selected meal recipe. The method may also include using the promotion processing server to determine a satisfaction level with the selected meal recipe based upon the facial image, generate a digital promotion toward another meal recipe having at least one common ingredient with the ingredients of the selected meal recipe based upon the satisfaction level, and communicate the digital promotion to the remote device.

A computer readable medium aspect is directed to a non-transitory computer readable medium for processing a promotion. The non-transitory computer readable medium includes computer executable instructions that when executed by a processor cause the processor to perform operations. The operations may include storing a plurality of meal recipes and associated ingredients and obtaining a selected meal recipe from a given user. The operations may also include cooperating with a camera of a remote device associated with the given user to obtain a facial image of the given user during consumption of ingredients of the selected meal recipe and determining a satisfaction level with the selected meal recipe based upon the facial image. The operations may also include generating a digital promotion toward another meal recipe having at least one common ingredient with the ingredients of the selected meal recipe based upon the satisfaction level, and communicating the digital promotion to the remote device.

DETAILED DESCRIPTION

Referring initially toFIGS. 1-2, a promotion processing system20includes a remote device30associated with a given user. The remote device30is illustratively in the form of a mobile wireless communications device and includes a housing31, a display32carried by the housing, and an input device33also carried by the housing. The display32and the input device33may be in the form of a touch display, as will be appreciated by those skilled in the art. The remote device30also includes a wireless transceiver34and a remote device controller35coupled to the wireless transceiver. The display32and input device33are also coupled to the remote device controller35.

A camera36is also carried by the housing31and is coupled to the remote device controller35. While the remote device30is illustratively in the form of a mobile wireless communications device, the remote device may include or be in the form of a personal computer, laptop computer, tablet computer, or wearable device.

The promotion processing system20also includes a promotion processing server40. The promotion processing server includes a processor41and an associated memory42. While operations of the promotion processing server40are described herein, it will be appreciated that the processor41and memory42cooperate to perform the operations.

Referring now to the flowchart60inFIG. 3, beginning at Block62, operations of the promotion processing server40will now be described. At Block64, the promotion processing server40stores meal recipes21a-21nand associated ingredients22a-22n, for example, in the memory42. For example, a meal recipe21a-21nmay include a recipe for an entire meal (e.g., a meal kit), or an individual meal item (e.g., grilled chicken). Similarly, the ingredients22a-22nmay be for an item in the meal kit (e.g., mashed potatoes) or an item for the individual meal item (e.g., salt, pepper, etc.). The meal recipes21a-21nand associated ingredients22a-22nmay be stored in the memory42along with a corresponding user. In other words, the given user may be associated with the meal recipes21a-21n, for example, and may have stored the meal recipes as a “saved recipe” or “liked recipe.” The meal recipes21a-21nmay be meal recipes that are of interest to the given user and not necessarily recipes that have been previously prepared and consumed by the given user.

The promotion processing server40, at Block66, obtains a selected meal recipe21bfrom the given user. For example, a listing of saved meal recipes21a-21nassociated with the given user may be displayed on the display32of the remote device30, and the user may select, by way of the input device33, the selected meal recipe21b. The selected meal recipe21bmay be one selected by the given user for preparation and consumption.

The promotion processing server40cooperates with the camera36of the remote device30to obtain one or more facial images27of the given user during consumption of ingredients22bof the selected meal recipe21b(Block68). In other words, the remote device30may be configured to capture still images or video of the given user's face as they eat the selected meal recipe21b.

At Block74, the promotion processing server40determines a satisfaction level with the selected meal recipe21bbased upon the facial image27. Those skilled in the art will appreciate that the given user's facial image27may be representative of the satisfaction level of the given user with the selected meal recipe21b. For example, if the given user does not like the selected meal recipe21b, the facial image27may indicate such, for example, the facial image may include a frown or other distorted feature indicative of a lower satisfaction level. Conversely, if the given user likes the selected meal recipe21b, the facial image27may indicate such, for example, the facial image may include a smile or other feature indicative of a higher satisfaction level.

The satisfaction level may conceptually be considered a score (e.g., 0 to 10) of the level of which the given user likes the selected meal recipe21b. For example, a higher score (e.g., 7-10) may be indicative that the given user more strongly likes the selected meal recipe21b, while a lower score (e.g., 0-3) may be indicative that the given user more strongly dislikes the selected meal recipe. A mid-range score (e.g. 4-6) may be indicative that the given user is ambivalent toward the selected meal recipe21b. Further details of determining the satisfaction level will be described in further detail below.

The promotion processing server40generates a digital promotion24toward another meal recipe25having at least one common ingredient with the ingredients22bof the selected meal recipe21bbased upon the satisfaction level (Block76). More particularly, the promotion processing server40may generate the digital promotion24when the satisfaction level is indicative of overall satisfaction with the selected meal recipe21b, for example, the given user likes the selected meal recipe. Alternatively, the promotion processing server40may generate the digital promotion24when the satisfaction level is indicative of overall dissatisfaction with the selected meal recipe21b. For example, the promotion processing server40may generate the digital promotion24as an incentive to purchase another meal recipe25since the given user was dissatisfied with the selected meal recipe21b.

The digital promotion24may be applicable to or redeemable toward the common ingredient of the meal recipes21b,25. For example, if the selected meal recipe21bis a meal kit that includes mashed potatoes, grilled chicken, and broccoli, the digital promotion24may be redeemable toward any of potatoes, chicken, or broccoli. In another example, where the selected meal recipe is teriyaki chicken, the digital promotion24may be redeemable toward teriyaki sauce. Of course the digital promotion24may be redeemable toward another ingredient in the another meal recipe25.

The another meal recipe25may be determined based upon the satisfaction level. For example, if the satisfaction level is indicative of dissatisfaction of the selected meal recipe21b, e.g., when the selected meal recipe is a meal kit that includes mashed potatoes, grilled chicken, and broccoli, the another meal recipe25may be determined to be a broccoli-based pasta meal kit. However, if the satisfaction level is indicative of overall satisfaction of the selected meal recipe21b, then the promotion processing server40may determine the another meal recipe25to be similar to that selected meal recipe (e.g., a grilled chicken based meal recipe).

In some embodiments, the promotion processing server40may generate the digital promotion24to have a redemption value that is based upon the satisfaction level. For example, if the promotion processing server40determines that the satisfaction level is indicative of the given user “loving” the selected meal recipe21b, the promotion processing server may generate the digital promotion24to have a higher redemption value. Of course, the promotion processing server40may generate the digital promotion24to have a redemption value that is lower with a higher satisfaction level. For example, if the given user “loves” the selected meal recipe21b, the given user may be more likely to repurchase the selected meal recipe without a promotion, and thus would receive a digital promotion24having a lower redemption value (e.g., $0.50 OFF).

At Block78, the promotion processing server40communicates the digital promotion24to the remote device30, for example, wirelessly. In some embodiments, the promotion processing server40may also communicate the another meal recipe25to the remote device30, e.g., along with the digital promotion. The digital promotion24and/or the another meal recipe25may be displayed on the display32of the remote device30. The digital promotion24may be stored in a digital wallet associated with the given user and/or displayed on the display32of the remote device30for redemption, for example, at a POS terminal. The another or new meal recipe25may be stored in the memory42, for example, and be associated with the given user so that the another meal recipe is user-selectable in the future, for example, to be indicative of planned consumption. Operations end at Block80.

Referring now toFIGS. 4-5, and the flowchart160inFIG. 6, beginning at Block162, more detailed operations of the promotion processing server40will now be described, particularly with respect to determining the satisfaction level. Blocks164-168are similar to Blocks64-68described above and need no further description.

At Block170, the promotion processing server40applies image processing to the facial image27or images to extract facial features. Exemplary facial features include visibility of teeth28, a mouth-opening distance29, and an eyebrow raise distance23. Of course other facial features may be extracted, for example, as described in the technical article entitled, “Facial expression (mood) recognition from facial images using committee neural networks” by Saket S. Kulkarni, Narender P. Reddy, and S I Hariharan, the entire contents of which are hereby incorporated by reference.

The promotion processing server40determines a facial expression of the given user during consumption of the ingredients based upon the facial features (Block172). More particularly, the promotion processing server40may determine the given user is smiling, for example, based upon frown lines and visibility of teeth28. In contrast, the promotion processing server40may determine the given user is unhappy based upon frown lines and lack of visible teeth28. Distances and other facial features can be mapped to determine a facial expression, for example, using a look-up table and/or machine learning from the given user or across multiple users (e.g., using regression algorithms).

At Block174, the promotion processing server40determines the satisfaction level based upon the facial expression. For example, the facial expression may be one of excitement rather than happiness, and thus the satisfaction level may be indicative of greater satisfaction. A larger amount of teeth28showing relative to other facial expressions may be indicative of a higher amount of satisfaction, for example.

In some embodiments, the promotion processing server40may use a relative position or a reference position to extract the satisfaction level. For example, the promotion processing server40may use changes in relative facial features or expressions when multiple facial images27of the given user are obtained during consumption of the selected meal recipe21b. If the given user typically, in a neutral facial expression has teeth showing, then the promotion processing server40may set that amount of teeth showing as a reference amount and determine the facial expression based upon changes in the amount of teeth showing relative to the reference amount. Of course, while the amount of teeth showing has been described, use of a reference position may be applicable to any extracted facial feature, as will be appreciated by those skilled in the art. Blocks176and178are similar to Block76and78described above and need no further description. Operations end at Block180.

A method aspect is directed to a method of processing a promotion. The method includes using promotion processing server40to store a plurality of meal recipes21a-21nand associated ingredients22a-22n, obtain a selected meal recipe21bfrom a given user, and cooperate with a camera36of a remote device30associated with the given user to obtain a facial image27of the given user during consumption of ingredients of the selected meal recipe. The method also includes using the promotion processing server40to determine a satisfaction level with the selected meal recipe21bbased upon the facial image27, generate a digital promotion24toward another meal recipe25having at least one common ingredient22bwith the ingredients22a-22nof the selected meal recipe21bbased upon the satisfaction level, and communicate the digital promotion to the remote device30.

A computer readable medium aspect is directed to a non-transitory computer readable medium for processing a promotion. The non-transitory computer readable medium includes computer executable instructions that when executed by a processor41cause the processor to perform operations. The operations include storing a plurality of meal recipes21a-21nand associated ingredients22a-22nand obtaining a selected meal recipe21bfrom a given user. The operations also include cooperating with a camera36of a remote device30associated with the given user to obtain a facial image37of the given user during consumption of ingredients of the selected meal recipe21band determining a satisfaction level with the selected meal recipe based upon the facial image. The operations also include generating a digital promotion24toward another meal recipe25having at least one common ingredient22bwith the ingredients22a-22nof the selected meal recipe21bbased upon the satisfaction level, and communicating the digital promotion to the remote device30.

While several embodiments have been described herein, it should be appreciated by those skilled in the art that any element or elements from one or more embodiments may be used with any other element or elements from any other embodiment or embodiments. Many modifications and other embodiments of the invention will come to the mind of one skilled in the art having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is understood that the invention is not to be limited to the specific embodiments disclosed, and that modifications and embodiments are intended to be included within the scope of the appended claims.