Patent Description:
Cooking apparatus may implement cooking processes such as roasting, grilling, frying, air frying, etc., to cook food. For example, cooking apparatus such as ovens or air fryers may be used for roasting food. In an example use case, a user may set the cooking temperature of the cooking apparatus while also defining the amount of time that the cooking apparatus is to cook the food (e.g., by setting a timer at which point the cooking apparatus turns off or the user manually removes the food from the cooking apparatus). However, the outcome of the cooking process (e.g., quality of the cooked food, time to cook the food, etc.) may depend on factors such as the user's skill or experience, how closely the user follows a recipe and/or the cooking efficiency of the cooking apparatus. In the example of roasting, a user may set the cooking apparatus to cook at a relatively high temperature, to roast the food within a specified time period. While this high temperature cooking process may cook the food within a relatively short period of time and/or result in food having a desirable flavor, there is a risk of an outcome such as the food being overcooked or less tasty than anticipated.

<CIT> describes a heating algorithm to cook at least one food substance in a cooking chamber, detect a state change of the food substance, and modify the heating algorithm to reconfigure the system resources supplied to the heating elements in response to the state change.

<CIT> describes a cooking appliance that uses machine learning models to provide automation of a cooking process.

<CIT> describes a weight sensor to monitor the weight of a processed food substance, and a control device to obtain weight information for the food substance, compute a derivative signal that is indicative of a weight loss rate of the food substance, compute a characteristic value of the derivative signal, and in an initial stage of a heating procedure, based on the characteristic value, determine an estimate for an initial fat content of the food substance, for the ongoing heating procedure, and/or an estimate for a resulting fat content change for the food substance.

<CIT> describes a mess detector for a cooking appliance having a cook surface with one or more heating zones, the mess detector comprising: a housing having a mount for securing to the cooking appliance or nearby structure within a line of sight of the cook surface; at least one sensor configured to sense a mess on the cook surface, the at least one sensor comprising a visual light imaging device and outputting image data; and a projector emitting a projected image indicative of a sensed mess onto the cook surface; and a controller comprising a processor receiving the image data and programmed with one or more algorithms to process the image data to determine when a mess has occurred on the cook surface and indicating that a mess is detected.

<CIT> describes a device for determining sugar content information for a food item during or after cooking.

Certain aspects or embodiments described herein may relate to improving the outcome of a cooking process and/or reducing burdens for a user.

Advantageous optional features are set out by the dependent claims.

In a first aspect, a computer-implemented cooking method is according to present claim <NUM>. The method comprises receiving first image data corresponding to a view associated with a solid food item at a first time of a cooking process implemented by a cooking apparatus. The method further comprises receiving second image data corresponding to a view associated with the food item at a second time of the cooking process. The method further comprises comparing the first image data with the second image data to detect a geometric change indicative of liquid loss of the food item between the first time and the second time as a result of the cooking process. In response to the detected geometric change being indicative of a transition from a first phase to a second phase of the cooking process where liquid loss from the food item as a result of the cooking process occurs at a higher rate during the second phase than the first phase, the method further comprises causing the cooking apparatus to modify the cooking by reducing a cooking temperature of the cooking apparatus.

Some embodiments relating to the first and other aspects are described below.

The transition from the first phase to the second phase occurs when, during the cooking process, a structure of the food item undergoes a physical change as a result of the cooking process such that liquid is released from the structure.

The method further comprises detecting the geometric change of the food item by identifying a change in position and/or size of at least part of the food item between the first and second image data.

The method further comprises identifying the transition from the first phase to the second phase. The transition is identified by comparing the identified change in position and/or size with a threshold geometrical change value indicative of the transition. In response to the identified change crossing the threshold geometrical change value, the cooking apparatus causes the cooking apparatus to modify the cooking process by reducing a cooking temperature of the cooking apparatus.

In some embodiments, the method comprises identifying the change in position. The change in position may be identified by: identifying a coordinate in the first image data that corresponds to a region of the food item; identifying a coordinate in the second image data that corresponds to the region of the food item; and identifying the change in position based on a change of the identified coordinate corresponding to the region of the food item between the first image data and the second image data.

In some embodiments, the method comprises identifying the change in size. The change in size may be identified by: identifying a set of coordinates in the first image data that are indicative of a dimension of the food item at the first time; identifying a set of coordinates in the second image data that are indicative of the dimension of the food item at the second time; and identifying the change in size based on a change of identified coordinates between the first and second image data.

In some embodiments, the indication of liquid loss comprises a change in an amount of visible liquid on a surface of the food item and/or on a support surface for the food item.

In some embodiments, the method comprises using a surface liquid detection model to detect, between the first and second image data, the change in the amount of visible liquid on the surface of the food item and/or on the support surface for the food item. The surface liquid detection model is configured to identify the transition from the first phase to the second phase. The transition may be identified by comparing the identified change in the amount of liquid on the surface of the food item and/or on the support surface with a threshold liquid change value indicative of the transition.

In some embodiments, the method comprises causing the cooking apparatus to modify the cooking process in response to the identified change crossing the threshold liquid change value.

In some embodiments, where there is a plurality of food items to be cooked, the method comprises causing the cooking apparatus to modify the cooking process in response to identifying when, during the cooking process, a predefined number or proportion of the plurality of food items are associated with the detected geometric change indicative of the transition from the first phase to the second phase of the cooking process.

In some embodiments, the method comprises receiving data indicative of a type of the food item received by the cooking apparatus for cooking. The method may further comprise causing the cooking apparatus to apply heat at a first target temperature. In response to the detected geometric change being indicative of the transition from the first phase to the second phase of the cooking process, the method may further comprise causing the cooking apparatus to apply heat at a second, reduced, target temperature. The indication of the geometric change detected during the cooking process may be compared with a predefined threshold change value based on the type of the food item. The predefined threshold change value may be indicative of the transition from the first phase to the second phase of the cooking process.

In a second aspect, a non-transitory machine readable medium is according to present claim <NUM>.

In a third aspect, a cooking apparatus is according to present claim <NUM>.

Certain aspects or embodiments described herein may provide various technical benefits such as reducing the amount of liquid lost from food during a cooking process, which may help to improve an outcome of the cooking process. In this manner, certain aspects or embodiments described herein may lead to an improved outcome such as improved food quality (e.g., food flavor, texture, juiciness, etc.), reduced cooking time compared with low temperature cooking processes and/or reduced cleaning or maintenance of cooking surfaces, among other benefits such as referred to herein.

Exemplary embodiments of the invention will now be described, by way of example only, with reference to the following drawings, in which:.

As referred to herein, a "cooking process" refers to applying heat to a food item to cause a change in the food item. Such application of heat may result in a mere warming of the food item, or a more substantial change in the food item such as may be achieved by using cooking methods such as roasting, grilling, frying, air frying, etc..

As referred to herein, a "cooking apparatus" refers to any device capable of applying heat to a food item, in order to complete the cooking process as referred to above. Heat may be applied to the food item by the cooking apparatus in one or multiple ways such as by conduction, convection or radiation. Examples of cooking apparatus include: an oven, microwave oven, hob, air fryer, etc..

Cooking apparatus such as air fryers may be conveniently used for roasting food such as meat, fish and vegetables. To create a delicious taste and reduce the cooking time, the roasting temperature may be set at <NUM>, or higher. Increasing the cooking temperature may reduce the time for the cooking process to complete. In addition, such high temperature roasting may result in the food having a desirable flavor. For example, a user may prefer the flavor of meat, fish or vegetables roasted at a high temperature compared with meat, fish or vegetables roasted at a lower temperature, or cooked using a different cooking process such as boiling.

However, food such as meat, fish and vegetables may have a high liquid content (e.g., water, lipids and/or proteins that may be released from the food in liquid form during the cooking process). Such food may comprise structures which, when heated, undergo physical changes resulting in the release of liquid during the cooking process. For example, muscle fibers may shrink or break, proteins may become denatured, collagen may dissolve, fats may melt, etc., as a result of the cooking process.

Raw meats and seafood may have high water content. Some of this water content may contribute to the "juiciness" of the meat. However, the cooking process may reduce the juiciness of the meat. For example, some water may be stored in spaces between protein filaments of the muscle fibers, as well as around the muscle fibers. During the cooking process, some proteins may denature such that the core of the muscle fibers may collapse and shrink. As the cooking process continues, juice (i.e., water in combination with any other components dissolved or otherwise mixed with the water such as sugars, salts, protein components, lipids, etc.) may be squeezed out from within and around the muscle fibers. The juice released by this process may combine with oil/fats released from other structures in the meat as a result of the cooking process. The liquid leached out from the meat may be considered "free" liquid. However, some "trapped" liquid may be left behind after completion of the cooking process. Trapped liquid may not necessarily provide the desired juiciness since it may be trapped by gel-form proteins or otherwise bound to various components of the cooked meat.

In an example, a significant amount of liquid may be released from the various structures of the meat as a result of the cooking process when the temperature of the meat exceeds <NUM>. As the cooking process continues, "free" liquid may be released, leaving behind "trapped" liquid. For example, by the time the meat reaches a temperature of <NUM>, collagen shrinkage may have released most of the "free" liquid leaving behind mostly "trapped" liquid. Therefore, the meat may be considered dry and/or tough at this point.

Similarly, structures in vegetables such as cells storing high-water content liquid may shrink or become broken during the cooking process to release the liquid therefrom, which may lead to vegetables shrinking and/or becoming dryer during the cooking process.

Thus, certain liquid released from food during the cooking process may make the food dry, hard, less juicy, less tasty, etc. Food (such as meat, vegetables, etc.) may be considered to be solid. However, the food may release liquid (e.g., leach-out liquid that may seep out from the food) during the cooking process.

A cooking process called "low temperature roasting" (where the cooking temperature may be lower than <NUM>) may be used to retain as much liquid as possible in the food (e.g., to reduce leaching out of the liquid). However, the low temperature roasting method increases the time to cook the food, compared with the high temperature roasting method. In addition, there may be cases where food cooked using the low temperature roasting method may be considered to be less tasty than food cooked using the high temperature roasting method. Similar issues may exist with other cooking processes such as grilling, barbequing, etc..

In some cases, certain liquid such as oil/fat released from the food may become baked on to a cooking surface as "grease". Over time, the accumulation of the baked-on grease may make cleaning rather difficult. There are many cleaning materials in the market and many tips on how to clean baked-on grease. However, this may not be convenient for a consumer due to the extra attention and time needed for consumers to clean the cooking surface.

The release of liquid (which may also be referred to as "leach-out") during the cooking process may therefore cause various issues such as relating to the final quality of the cooked food and/or burdens for a user implementing the cooking process.

<FIG> refers to a cooking method <NUM> (i.e., a computer-implemented method) according to an embodiment. The cooking method <NUM> may be used for controlling a cooking process implemented by a cooking apparatus. The cooking method <NUM> is computer-implemented (e.g., using processing circuitry of an entity such as referred to in certain systems described below).

The cooking method <NUM> comprises, at block <NUM>, receiving first image data corresponding to a view associated with a food item at a first time of a cooking process implemented by a cooking apparatus.

The cooking method <NUM> further comprises, at block <NUM>, receiving second image data corresponding to a view associated with the food item at a second (later) time of the cooking process.

In some cases, the first and/or second image data may correspond to or may be derived from imaging data captured by a camera (e.g., a camera of the cooking apparatus itself or a camera as a separate entity to the cooking apparatus). Such a camera may be positioned inside the cooking apparatus itself or outside the cooking apparatus but with a region of interest associated with the food item in a "field of view" of the camera.

In some cases, such first and/or second image data may correspond to the raw imaging data captured by the camera. In some cases, such first and/or second image data may comprise processed imaging data (e.g., compressed imaging data, enhanced imaging data, etc., derived from the raw imaging data).

The "view associated with the food item" may refer to a region of interest associated with the food item in a "field of view" of the camera. In some cases, at least part of the food item may be visible in the view of the camera. In some cases, the food item may not be visible in the view. In this latter case, the "region of interest" may refer to an area around the food item which may be of interest for various reasons as described herein. In either case, the first and second image data may provide information about the food item at the first and second times, respectively, which may be used in the cooking method <NUM> as further described below.

The cooking method <NUM> further comprises, at block <NUM>, comparing the first image data with the second image data to detect a geometric change of the food item between the first time and the second time as a result of the cooking process.

In response to the detected geometric change being indicative of a transition from a first phase to a second phase of the cooking process where liquid loss from the food item as a result of the cooking process occurs at a higher rate during the second phase than the first phase, the cooking method <NUM> further comprises, at block <NUM>, causing the cooking apparatus to modify the cooking process.

As highlighted above, the cooking process may lead to various physical changes in the food item leading to liquid loss from the food item (and a corresponding geometric change of the food item). The comparison of the first and second image data may yield information about a physical change of the food item that may provide an indication of the liquid loss that corresponds to the geometric change. A geometric change of the food item between the first and second times may provide such an indication of liquid loss. In some embodiments, visible leached-out liquid from the food item may provide the indication of the liquid loss and corresponding geometric change of the food item.

The timing of the transition from the first phase to the second phase may depend on various factors such as the type of food item being cooked, efficiency of the cooking process, size of the food, cooking recipe being followed, etc. Further, the transition from the first phase to the second phase may not be instantaneous. For example, different parts of the food item may reach a certain temperature at different times (e.g., the core of the food item may take longer to cook than the surface of the food item). Further, different parts of the food item may respond to heat differently (e.g., predominantly fatty structures may release liquid at a different temperature to predominantly water and/or protein-based structures).

For a period of time after the cooking process has started, there may be little or no liquid loss. The "first phase" may comprise this period of time with little or no liquid loss. During the first phase, the core temperature of the food item may be relatively low such that the structures in the core of the food item do not undergo any substantial physical changes leading to the release of liquid therefrom.

However, as the food item cooks, various physical changes may occur which may be associated with an increase in the rate of liquid loss. This increased rate of liquid loss may occur during the "second phase". There may still be some liquid loss during the first phase. However, the rate of liquid loss in the second phase may be substantially increased compared with the first phase. After the second phase, a further "third" phase may occur where the rate of liquid loss decreases again (e.g., when most of the free liquid has been released to leave behind the trapped liquid).

Some example models (implemented by the processing circuitry as part of a cooking method) for identifying the transition based on the comparison are described below. In each example model, the comparison of the first and second image data provides information indicative of the rate of liquid loss. The identification of the transition is based on identifying when, during the process, the rate of liquid loss (and the corresponding geometric change) is indicative of a threshold being crossed where the threshold is indicative of the transition.

Such a threshold may depend on a cooking factor such as the type of food item being cooked, efficiency of the cooking process, size of the food, cooking recipe being followed, etc. In some cases, a cooking study may have been carried out to determine the rate of liquid loss over time for a certain type of food item, as well as establishing how this liquid loss affects the final quality of the food item. Thus, the cooking study may provide information for use in selecting the threshold. However, in some cases, a study may not be used for selecting the threshold. For example, some models may regard the transition as occurring when the indication of liquid loss is first detected during the cooking process (thus the threshold is when liquid loss is first detected). In another example, some models may regard the transition as occurring based on when a physical change is identified and quantified, such as a reduction in the size of the food item by a predefined threshold.

In an example model, a comparison of the first and second image data may yield information indicating a proportion of liquid has been lost from the food item as a result of the cooking process. The example model may be configured to determine (e.g., estimate, quantify, etc.) a proportion of the liquid that has been lost from the food item between the first and second times. Various ways this information can be established are described below. The identification of the transition may be based on when the comparison indicates that a threshold proportion of liquid has been lost from the food item. As highlighted above, the threshold may be based on a cooking factor.

In another example model, if the amount of liquid lost can be quantified within a set of equal time intervals, it may be possible to detect when the transition occurs based on when the rate of detected liquid loss crosses the threshold. In this regard, the example model may be configured to determine (e.g., estimate, quantify, etc.) how much liquid has been lost from the food item during each time interval. A comparison of first and second image data associated with a first time interval may yield information indicating an estimate or other indication of an amount of liquid lost during the first time interval. A comparison of first and second image data associated with a second, subsequent, time interval may yield information indicating an estimate or other indication of an amount of liquid lost during the second time interval. If the amount of liquid lost is the same or does not significantly increase in each of the first and second time intervals, then this may indicate that the transition has not occurred. However, if the amount of liquid lost increases in the second interval (e.g., increases by a threshold value), then this may indicate that the transition has occurred.

In another example model, a comparison of the first and second image data may indicate that a specified part of the food item (e.g., the core of the food item) has reached a threshold temperature associated with the transition. In this regard, the example model may be configured to identify the transition based on predetermined information indicating a relationship (e.g., based on a cooking factor) between an expected core temperature for a specified degree of liquid loss. In other words, the predetermined information may be used by the model to estimate the core temperature of the food item based on the amount liquid loss detected.

In these example models, a computer-based visual analysis of the first and second imaging data may be used to detect the indication of liquid loss. In response to the detected liquid loss (and corresponding geometric change) being indicative of the transition from the first phase to the second phase, the cooking process may be modified.

The modification of the cooking process involves reducing the cooking temperature. For example, the high temperature roasting method may be used in the first phase. Subsequently, a lower temperature roasting method may be used in the second phase.

A further possible modification of the cooking process may involve another change that may reduce the rate of liquid lost in the second phase. For example, a change of any setting of the cooking apparatus (such as a change of air circulation speed, etc.) that may otherwise slow down the cooking process may be used.

The cooking method <NUM> therefore causes the cooking apparatus to modify the cooking process by reducing a cooking temperature of the apparatus. In this regard, the cooking method <NUM> may cause a controller of the cooking apparatus to change its setting. For example, the cooking method <NUM> may cause a signal to be sent to the controller to cause the controller to e.g., reduce the cooking temperature.

The implementation of certain embodiments described herein such as the cooking method <NUM> may address at least one of the various issues described above. For example, the cooking method <NUM> may reduce the amount of liquid lost from food during a cooking process e.g., compared with some high temperature methods which may otherwise lead to excessive loss of liquid during the cooking process. Certain embodiments described herein may lead to an improved outcome such as improved food quality (e.g., food flavor, texture, juiciness, etc.), reduced cooking time compared with low temperature cooking processes and/or reduced cleaning or maintenance of cooking surfaces.

<FIG> is a schematic drawing of a cooking ecosystem <NUM> according to an embodiment. Certain embodiments described herein (e.g., cooking method <NUM>) may be implemented in certain parts of the cooking ecosystem <NUM>. The cooking ecosystem <NUM> depicts various devices and entities which may be deployed as part of the cooking ecosystem <NUM>. Not every device or entity depicted may be needed in some scenarios, as explained below.

The cooking ecosystem <NUM> comprises a cooking apparatus <NUM> for cooking a food item <NUM>. The cooking apparatus <NUM> comprises a controller <NUM> for controlling the cooking process. For example, the controller <NUM> may control a heating element (not shown) of the cooking apparatus <NUM> (e.g., to control the cooking temperature of the cooking apparatus <NUM>). The controller <NUM> is communicatively coupled to a camera <NUM> for capturing images. The camera <NUM> is positioned such that a region of interest associated with the food item <NUM> is within a field of view of the camera <NUM>. This particular configuration is an example. For example, as already explained, the camera <NUM> may or may not be inside the cooking apparatus <NUM>.

In some cases, the cooking ecosystem <NUM> comprises a cloud computing service <NUM> communicatively coupled to the controller <NUM>. A cloud computing service <NUM> may provide data storage and/or data processing services. The cloud computing service <NUM> may provide computing resource where there is insufficient computing resource available in any connected devices. In some cases, the cloud computing service <NUM> may provide updates and other services for the cooking apparatus <NUM>.

In some cases, the cooking ecosystem <NUM> comprises a user equipment <NUM> communicatively coupled to the controller <NUM>. A user equipment <NUM> may refer to any computing device associated with a user (e.g., of the cooking apparatus <NUM>). Examples of user equipment <NUM> include: a smartphone, smartwatch, tablet, Internet of Things (IoT) device, etc. In some cases, the user equipment <NUM> may be communicatively coupled to the cloud computing service <NUM>.

Any one or combination of the controller <NUM>, cloud computing service <NUM> and the user equipment <NUM> may be used to implement the cooking method <NUM> and other embodiments described herein. For example, in some cases, the controller <NUM> may implement the cooking method <NUM> and related embodiments. In this regard, the controller <NUM> may comprise processing circuitry (not shown) for implementing the cooking method <NUM> and related embodiments. In other cases, processing circuitry associated with the various devices and entities of the cooking ecosystem <NUM> may implement the cooking method <NUM> and related embodiments.

<FIG> is a schematic drawing of a cooking apparatus <NUM> for implementing a cooking process according to an embodiment. The cooking apparatus <NUM> according to the invention is that of appended claim <NUM>.

The cooking apparatus <NUM> comprises a cooking chamber <NUM> for receiving a food item <NUM>. The cooking apparatus <NUM> further comprises a housing <NUM> defining the cooking chamber <NUM>. The cooking apparatus <NUM> further comprises an air circulation system <NUM> for circulating air flow inside the cooking chamber <NUM>. Therefore, in this regard, the cooking apparatus <NUM> may have a similar form to a fan oven or an air fryer. The cooking apparatus <NUM> further comprises a camera <NUM> for capturing images (of the "view" associated with the food item <NUM>) during the cooking process. The captured images may correspond to or be used to derive the first and second image data.

The cooking apparatus <NUM> further comprises a controller <NUM> corresponding to the controller <NUM> of <FIG>. The controller <NUM> is configured to implement the cooking method <NUM>.

Thus, the controller <NUM> is configured to receive first image data corresponding to a view associated with a food item <NUM> at a first time of a cooking process implemented by the cooking apparatus <NUM> (in accordance with block <NUM>).

The controller <NUM> is further configured to receive second image data corresponding to a view associated with the food item <NUM> at a second time of the cooking process (in accordance with block <NUM>).

The controller <NUM> is further configured to compare the first image data with the second image data to detect a geometric change of the food item <NUM> between the first time and the second time as a result of the cooking process (in accordance with block <NUM>).

In response to the detected liquid loss being indicative of a transition from a first phase to a second phase of the cooking process where liquid loss from the food item <NUM> as a result of the cooking process occurs at a higher rate during the second phase than the first phase, the controller <NUM> is further configured to cause the cooking apparatus <NUM> to modify the cooking process (in accordance with block <NUM>).

<FIG> is a graph representative of a cooking process implemented according to a preferred embodiment. In the first phase of the cooking process, a cooking apparatus such as described above, is set to cook at a first temperature (for example, <NUM>). Upon identification of the "transition" as described herein (at the start of the "second phase"), the cooking process is modified by reducing the cooking temperature to a second, lower, temperature (for example, <NUM>). This cooking process may facilitate an initial high temperature cooking phase (e.g., to reduce overall cooking time, reduce the rate of food shrinkage and/or improve taste) while reducing the effect of certain issues arising from excessive liquid leach-out (e.g., reduced juiciness, excessive shrinkage, etc.) or burdens for the user (e.g., having to clean oil/fat spillage).

In some cases, the first phase may be referred to as a "no leach-out" phase and the second phase may be referred to as a "leach-out" phase. The transition may be detected based on a "smart" visual method, in accordance with various embodiments described herein.

Various implementations of the cooking method <NUM> and related embodiments are described below in the context of the cooking method of roasting. Similar principles may apply to other cooking methods.

The roasting procedure may be differentiated based on a "leach-out status" of the food item. In some cases, there are at least two phases: a "no leach-out roasting phase" (the first phase) and a "leach-out roasting phase" (the second phase).

The transition of these roasting phases may be made dynamically by processing circuitry (e.g., implementing a "smart" leach-out phase detection module) for implementing the cooking method <NUM>. For example, the processing circuitry may be configured to detect the start of the leach-out phase based on the images captured by the camera <NUM>.

In some cases, the processing circuitry may be configured to detect a visual geometry change or change rate of the food item <NUM> based on image/video analysis (e.g., based on the comparison of the first and second image data) in the first phase of the cooking process (for example, within a period of time <NUM> to <NUM> minutes for some food items). In this first phase, the cooking strategy is suitable for the "no leach-out roasting phase".

When the change in geometry or a change rate reaches a threshold (the threshold may be pre-set for a model, as referred to previously), the processing circuitry causes the cooking apparatus <NUM> to switch to the cooking strategy for the "leach-out roasting phase". For the "leach-out roasting phase", the cooking apparatus <NUM> applies a lower cooking temperature than the initial roasting phase (e.g., at least <NUM> lower than the cooking temperature used for the "no leach-out roasting phase").

The timing of the transition (i.e., when the "leach out roasting phase" starts) may be a parameter used by the cooking apparatus <NUM> to judge when the food item <NUM> may be completely cooked. For example, it may take a further specified period of time after the transition to complete roasting of the food item <NUM>. The cooking apparatus <NUM> may then reduce the temperature or turn off to slow down or stop the cooking process.

In one possible implementation, the comparison of the first and second image data yields information regarding geometry changes of the food item <NUM>. Such geometry changes provide the indication of the liquid loss. A threshold geometry change may be indicative of the transition. In this implementation, a detection model (implemented by the processing circuitry) may detect the position of food item's four external "tip" points (e.g., the top, bottom, left and right maximum external tip point within the field of view of the camera <NUM>). Within equal time intervals (e.g., every <NUM> minute), the comparison of the image data may provide information indicative of any changes to the position of the four external tip points (e.g., by tracking the position of the tip points within the field of view). After a certain period, e.g., <NUM> time intervals of detection, the detection model may select which of these tip points has moved by the most (indicative of shrinkage of the food item <NUM>). The detection model may continue detecting movement of the selected tip point for each time interval. When the change in position of the tip interval reaches a threshold for this type of food item <NUM> (and/or based on another food factor), this indicates that the leach-out roasting phase has started (i.e., the transition has occurred). Further details of this possible implementation are described below.

In additional possible implementation, the comparison of the first and second image data may yield information regarding a visible area change (e.g., in term of pixels) of the food item <NUM> within each time interval. For example, a detection model (implemented by the processing circuitry) may be configured to detect the area of the food item <NUM> (e.g., based on a segmentation model or other approach). A change of the area may be indicative of shrinkage, and hence provide the indication of liquid loss. A threshold area change specified by the model for the food item <NUM> may indicate when the transition occurs during the cooking process. Further details of this possible implementation are described below.

In additional possible implementation, the comparison of the first and second image data may yield information regarding visible leach-out liquid. For example, a detection model (implemented by the processing circuitry) may be configured to detect the leach-out liquid that is visible to the camera <NUM> (e.g., based on a segmentation model or other approach such as monitoring for changes in reflectance in a region of interest). In this regard, there may be visible liquid on the surface of the food item <NUM> itself or around the food item <NUM> on a "support surface" such as a baking tray, plate or a layer of aluminum foil under the food item <NUM>. Depending on the detection model, the comparison may be based on a part of the image data corresponding to the food item <NUM> itself or in a region of interest around the food item <NUM>. This implementation may be standalone or may be used to assist other implementations such as described above. Further details of this possible implementation are described below.

If there are a plurality of food items <NUM> in the cooking apparatus <NUM>, various different cooking strategies may be used to identify the transition (since the transition may occur at different times for different food items <NUM>). In some cases, the strategy may comprise regarding the transition as occurring when detecting the first food item <NUM> that has a geometry change that crosses the threshold. In some cases, the strategy may comprise regarding the transition as occurring when detecting the second one, third one, final one etc., of the food items <NUM> that each have a geometry change that crosses the threshold.

Some embodiments relating to the above are now described.

In some embodiments, in the first phase, liquid is retained by the food item <NUM>. In the second phase, liquid is leached out by the food item <NUM>. In such embodiments, there may be a distinct point where the start of liquid leach out is detected, after which the cooking process may be modified. In other embodiments, there may not be such a distinct change. For example, the rate of change of liquid loss may substantially increase in the second phase compared with the first phase. However, some liquid loss may still occur in the first phase.

The transition from the first phase to the second phase occurs when, during the cooking process, a structure of the food item <NUM> undergoes a physical change as a result of the cooking process such that liquid is released from the structure. The "structure" could be any region of the food item <NUM> storing matter that may undergo a change resulting in release of liquid during the cooking process, such as described above.

<FIG> schematically depict images <NUM> of a food item <NUM> during a cooking process implemented by a cooking apparatus such as described above. In <FIG>, the food item <NUM> is bounded by a rectangular box <NUM> fitted to the maximum tip points (top, bottom, left, right) of the food item <NUM> at the first and second times, respectively.

In this case, the coordinate of the left tip point is represented by the coordinate (x0, y0) in <FIG> and coordinate (x1, y1) in <FIG>. It can be seen that due to the particular shape of the food item <NUM>, this region of the food item <NUM> (defined by the coordinate) can be used as a reference for determining how the geometry of the food item <NUM> has changed. As depicted, shrinkage of the food item <NUM> as a result of the cooking process has caused the region of the food item <NUM> to move within the field of view. Since the coordinates may be determined from the pixel identifiers, it is possible to determine by how many pixels the region has moved. A threshold number of pixels (or distance) can be set for the food item <NUM> such that when the threshold is crossed, this may indicate occurrence of the transition.

<FIG> also indicate another way to determine the indication of liquid loss. In this regard, the width (w0) and height (h0) at the first time can be determined based on the dimensions of the rectangular box <NUM> in <FIG>. The width (w1) and height (h1) at the second time can be determined based on the dimensions of the rectangular box <NUM> in <FIG>. Thus, a change in width and/or height of the rectangular box <NUM> may be determined from the comparison. If the change crosses a predefined threshold for the food item <NUM>, this may indicate occurrence of the transition.

Another way to determine the indication of liquid loss based on the images <NUM> depicted by <FIG> may be to compare the area of the food item <NUM> at the first and second times. A threshold area may be defined to indicate occurrence of the transition.

Some of these methods may be computationally simple/light-weight to implement. For example, a relatively light-weight detection model may be used to detect the tip points or size of the rectangular box <NUM>. In another example, a segmentation model may be implemented to distinguish between pixels corresponding to the food item <NUM> and non-food item pixels.

In some cases, any of these detection models could be implemented by a machine learning (ML) model (e.g., trained to detect the tip points, perform segmentation, etc.).

In some cases, any of these detection models could be implemented by a deterministic, classical or non-ML model (e.g., based on analyzing pixel intensity/color values across the images where a step change in pixel intensity/color value may indicate an edge of the food item <NUM>).

<FIG> is a flowchart <NUM> referring to a computer-implemented cooking method for detecting the transition according to an embodiment. The flowchart <NUM> refers to an implementation similar to or corresponding to the detection model described above where a change in coordinate of a food item tip point may be used to provide the indication of the liquid loss. In this regard, the processing circuitry referred to previously may implement the flowchart <NUM>. As described in more detail below, the flowchart <NUM> refers to blocks implementing the functionality of certain embodiments described herein such as the cooking method <NUM> and certain related embodiments. In some embodiments, certain blocks may be omitted or implemented in a different order to that depicted, as described in more detail below.

The flowchart <NUM> starts at block <NUM>.

In some embodiments, at block <NUM>, the processing circuitry may receive the information about the food item <NUM> and/or a recipe to be followed for the cooking process. Such information may be input by a consumer (e.g., via the cooking apparatus <NUM>, <NUM> itself or the user equipment <NUM>) or by an automatic food recognition function based on the captured images. The information may not always be needed. For example, the functionality of certain embodiments may still be implemented without such information.

At block <NUM>, some initial parameters may be set e.g., based on certain cooking factors such as the type of food item/recipe to be followed. Examples of initial parameters that could be set include at least one, and any combination, of the following parameters A to D. Parameter "A" may be t = <NUM>, where "t" is cooking time. Parameter "B" may be "t_interval", the time interval for capturing images or video clips to analysis food geometry change. For example, t_interval may be set at <NUM> minute or any other appropriate time. In this example, a new image of the food item <NUM> captured every <NUM> minute may be received. Each new image may be compared with the preceding image and/or the first-captured image to detect any indication of liquid loss from the food item <NUM>. Parameter "C" may be "d_threshold", a distance threshold used for determining whether a geometry change of the food item <NUM> compared with its initial geometry is indicative of occurrence of the transition. The distance threshold may be based on a cooking factor such as the type of food item <NUM>. Parameter D may be "t", the heating temperature for the first phase. The heating temperature could be set by the consumer or by the cooking apparatus (e.g., a predefined temperature for the food item <NUM> being cooked).

At block <NUM>, images of the food item <NUM> are captured at time "t" at each "t_interval". Such images may correspond to or be used to derive the first and second image data referred to in the cooking method <NUM>. The "first image data" may refer to an image captured prior to capture of an image corresponding to the second image data. Depending on the model, the "comparison" may be made between images acquired at regular intervals or may be made between an image captured at time, t, with an image captured at time t=<NUM>.

At block <NUM>, the food item <NUM> is segmented from the background (e.g., using a segmentation model).

At block <NUM>, an edge of the food item <NUM> is detected by image processing (e.g., using an edge detection model). The edge detection model may be configured to detect the "tip points" of the food item <NUM> based on the information in the image data. For example, the region of the detected food item <NUM> closest to the top of the image frame may be regarded as the "top" tip point. A similar principle may be applied to the "bottom", "left" and "right" tip points with respect to the bottom, left and right of the image frame, respectively.

At block <NUM>, a frame corresponding to the rectangular box <NUM> intersecting each of the tip points is generated. The position of the corners (top-left, top-right, bottom-left and bottom-right) of the rectangular box <NUM> may be defined by x-axis ("X") and y-axis ("Y") coordinates as follows: X_top-left, Y_top-right, X_bottom-left, Y_bottom-right. With reference to <FIG>, it can be recognized that these coordinates may be used to determine the geometry change of the food item <NUM> between the first and second times.

At block <NUM>, a check is made whether the food item <NUM> is raw (i.e., if t=<NUM>) or is not raw/cooking has started (i.e., t><NUM>).

If the food item <NUM> is raw (i.e., at t=<NUM>, or "yes"), the flowchart <NUM> proceeds to block <NUM> where the values of the x- and y-axis coordinates described above may define a reference frame to be used as benchmark to calculate the geometry change as the cooking process continues.

If the food item <NUM> is not raw or cooking has started (i.e., t><NUM>, or "no"), at block <NUM>, the present time, t, is added with the parameter, "t_interval".

At block <NUM>, the change in coordinate corresponding to a distance "d" is calculated. For example, the detected distance "d" may correspond to the maximum distance between (x1,y1) and (x0,y0) in <FIG>. Since the coordinates may be based on pixel values, this may not correspond to an actual distance moved by the region of the food item <NUM>. Rather, the distance "d" may represent a magnified or de-magnified value with respect to the actual distance moved, depending on the set-up of the camera <NUM>.

At block <NUM>, at determination is made regarding whether or not "d_threshold" has been crossed.

If the detected distance "d" crosses (e.g., meets or exceeds) "d_threshold" (i.e., "yes"), the flowchart <NUM> proceeds to block <NUM> where the controller <NUM> may cause the cooking apparatus <NUM> to modify the cooking process (for the second phase), whereupon the flowchart <NUM> ends at block <NUM>.

However, if the detected distance "d" does not cross "d_threshold" (i.e., "no"), the flowchart <NUM> proceeds to block <NUM> (e.g., to capture a further image at the next time interval).

An example implementation of the flowchart <NUM> is now described in an experiment that involved roasting salmon. During the experiment, the salmon shrunk during the first phase of the cooking process, during which the salmon was roasted at <NUM>. It was observed that the "tip points" of the salmon changed position during roasting. Based on detecting the pixel coordinates corresponding to each tip point of the salmon, it was observed that the movement of the tip point exceeded the predefined threshold distance of <NUM> pixels after <NUM> minutes of roasting. This threshold distance was considered to represent the start of the second phase. The cooking process was then automatically modified for the second phase by reducing the heating temperature to <NUM>. The cooking process continued for <NUM> minutes until the salmon was cooked. The length of time of the second phase could be based on a predefined time or could be controlled based on the detecting a threshold color change (e.g., browning of the surface of the salmon).

In a similar experiment that involved roasting chicken wings, the initial temperature was <NUM> for the first phase. Similarly, it was observed that the movement of one of the tip points of the chicken exceeded the predefined threshold distance of <NUM> pixels after <NUM> minutes of roasting. In this experiment, multiple chicken wings were cooked. The first chicken wing that had a tip point that exceeded the threshold distance was considered to provide the time of the transition. The cooking process was then automatically modified for the second phase by reducing the heating temperature to <NUM>. The cooking process continued for <NUM> minutes until the chicken was cooked. Again, the length of time of the second phase could be based on a predefined time or could be controlled based on the detecting a threshold color change (e.g., browning of the surface of the chicken).

<FIG> refers to a computer-implemented cooking method <NUM> according to an embodiment. The cooking method <NUM> is based on where the indication of liquid loss comprises a geometric change of the food item <NUM> (e.g., such as described in relation to <FIG>, to which reference may be made). The cooking method <NUM> may be implemented by the same processing circuitry for implementing the cooking method <NUM> and any related embodiments. In some embodiments, certain blocks of the cooking method <NUM> may be omitted or performed in a different order to that shown, as described in more detail below.

The cooking method <NUM> comprises, at block <NUM>, detecting the geometric change of the food item <NUM> by identifying a change in position and/or size of at least part of the food item between the first and second image data. In other words, the geometric change may be detected by comparing the first and second image data to determine a change in a coordinate or a set of coordinates (or a pixel or set of pixels) corresponding to (e.g., mapped to) at least part of the food item <NUM> between the first time and second time. The part of the food item may be the same part of the food item between the images such as an identified edge or corner (or other distinct feature) of the food item <NUM> visible in the view.

The cooking method <NUM> comprises, at block <NUM>, identifying the transition from the first phase to the second phase by comparing the identified change in position and/or size with a threshold geometrical change value (e.g., "d_threshold" or a threshold width or height of the food item <NUM>) indicative of the transition. In response to the identified change crossing the threshold geometrical change value, the cooking method <NUM> further causes the cooking apparatus to modify the cooking process.

In some embodiments, the cooking method <NUM> comprises, at block <NUM>, identifying the change in position by: identifying a coordinate in the first image data that corresponds to a region of the food item; identifying a coordinate in the second image data that corresponds to the region of the food item <NUM>; and identifying the change in position based on a change of the identified coordinate corresponding to the region of the food item <NUM> between the first image data and the second image data. Block <NUM> may refer to tracking movement of at least one of the tip points referred to in <FIG>. An identified coordinate of a tip point may correspond to a position of a "max point" in the food contour. The coordinate may correspond to a pixel identifier (e.g., a pixel location) in the image data.

In some embodiments, the cooking method <NUM> comprises, at block <NUM>, identifying the change in size by: identifying a set of coordinates in the first image data that are indicative of a dimension of the food item <NUM> at the first time; identifying a set of coordinates in the second image data that are indicative of the dimension of the food item <NUM> at the second time; and identifying the change in size based on a change of identified coordinates between the first and second image data. "A" dimension may refer to a length/width/height or it could refer to a plurality of dimensions e.g., area or volume. The change may be a change in number of identified coordinates corresponding to the dimension of the food item <NUM>.

Thus, in some cases, block <NUM> may refer to identifying a change in width, height, depth, area or volume of the food item <NUM>.

In one implementation, the "set of coordinates" may refer to at least two coordinates (e.g., X_top-left, Y_top-right, X_bottom-left, Y_bottom-right, etc.) that are indicative of a dimension such as length or width.

In another implementation, the "set of coordinates" may refer to each of the pixel coordinates between two opposite edges of the food item <NUM>. For example, the maximum distance between the same two opposite edges may be <NUM> pixels at the first time and <NUM> pixels at the second time. Thus, the number of identified coordinates has changed between the first and second times.

In another implementation, the "set of coordinates" may refer to the pixels making up the visible area of the food item <NUM>. For example, the total number of pixels corresponding to the area may be <NUM> pixels at the first time and <NUM> pixels at the second time. Thus, the number of identified coordinates has changed between the first and second times.

<FIG> schematically depict images <NUM> of a food item <NUM> during a cooking process implemented by a cooking apparatus such as described above. In <FIG>, the food item <NUM> comprises a food surface region of interest <NUM> in <FIG> respectively. As further shown by <FIG>, leached-out liquid <NUM> is visible on a support surface for supporting the food item <NUM>. Liquid loss may be detected in either or both of the food surface region of interest <NUM> and on the support surface. Thus, in some embodiments, the indication of liquid loss comprises a change in an amount of visible liquid on a surface of the food item <NUM> and/or on a support surface for the food item <NUM>. Certain embodiments may make use of this detectable liquid loss to identify the transition, as described in more detail below.

<FIG> refers to a computer-implemented cooking method <NUM> according to an embodiment. The cooking method <NUM> may be based on embodiments where the indication of liquid loss comprises a change in an amount of visible liquid on a surface of the food item <NUM> and/or on a support surface for the food item <NUM> (e.g., such as described in relation to <FIG>, to which reference may be made). The cooking method <NUM> may be implemented by the same processing circuitry for implementing the cooking method <NUM> and any related embodiments. In some embodiments, certain blocks of the cooking method <NUM> may be omitted or performed in a different order to that shown, as described in more detail below.

In some embodiments, the cooking method <NUM> comprises, at block <NUM>, using a surface liquid detection model to detect, between the first and second image data, the change in the amount of visible liquid on the surface of the food item <NUM> and/or on the support surface for the food item <NUM>.

The surface liquid detection model may be configured to detect/recognize visible liquid such as based on a change in reflectance or color in a detection area (on the food item <NUM> itself such as within the food surface region of interest <NUM> or on the support surface). This could be based on a deterministic method for detecting a change (based on image processing, detecting a change in color values, histogram analysis, etc.) or an ML-based method (e.g., trained to recognize a change in visible liquid such as based on a segmentation model).

In some embodiments, the surface liquid detection model is configured to identify the transition from the first phase to the second phase by comparing the identified change in the amount of liquid on the surface of the food item and/or on the support surface with a threshold liquid change value (e.g., a predefined value for the model) indicative of the transition. The cooking method <NUM> may further comprise, at block <NUM>, causing the cooking apparatus to modify the cooking process in response to the identified change crossing the threshold liquid change value. In some cases, the transition may be identified as soon as visible liquid is detected by the surface liquid detection model. In some cases, the transition may be identified when a threshold amount of visible liquid is detected by the surface liquid detection model (e.g., based on a change in area, reflectance and/or color, etc.).

<FIG> refers to a computer-implemented cooking method <NUM> according to an embodiment. The cooking method <NUM> may be implemented by the same processing circuitry for implementing the cooking method <NUM> and any related embodiments.

In the cooking method <NUM>, where there is a plurality of food items <NUM> to be cooked, a decision is made regarding the timing of the transition in view of the different cooking times of the different food items <NUM>. In this regard, the cooking method <NUM> comprises, at block <NUM>, causing the cooking apparatus <NUM> to modify the cooking process in response to identifying when, during the cooking process, a predefined number or proportion of the plurality of food items <NUM> are associated with the detected liquid loss (e.g., corresponding to the detected geometric change) indicative of the transition from the first phase to the second phase of the cooking process. The predefined number or proportion may refer to the first food item <NUM> that undergoes the transition, or a subsequent food item <NUM> undergoes the transition. The selection of the predefined number or proportion may be based on a previous study regarding which predefined number or proportion delivers an optimum outcome (e.g., reduced liquid loss, maximum flavor, reduced cooking time, etc.).

<FIG> refers to a computer-implemented cooking method <NUM> according to an embodiment. The cooking method <NUM> may be implemented by the same processing circuitry for implementing the cooking method <NUM> and any related embodiments. In the cooking method <NUM>, the type of food item <NUM> being cooked is taken into account to control the cooking process.

The cooking method <NUM> comprises, at block <NUM>, receiving data indicative of a type of the food item <NUM> received by the cooking apparatus <NUM> for cooking.

The cooking method <NUM> further comprises, at block <NUM>, causing the cooking apparatus <NUM> to apply heat at a first target temperature (e.g., for the first phase). The first target temperature may be based on the type of the food item and/or any other cooking factor such as described above.

In response to the detected liquid loss (e.g., corresponding to the detected geometric change) being indicative of the transition from the first phase to the second phase of the cooking process, the cooking method <NUM> further comprises causing the cooking apparatus <NUM> to apply heat at a second, reduced, target temperature (e.g., for the second phase).

The indication of the liquid loss (e.g., corresponding to the detected geometric change) detected during the cooking process is compared with a predefined threshold change value based on the type of the food item. The predefined threshold change value may be indicative of the transition from the first phase to the second phase of the cooking process.

The "predefined threshold change value" may correspond to the "threshold geometrical change value" and/or the "threshold liquid change value" referred to above. The target temperature(s) may be: user-specified, predefined based on the type of food item and/or based on a desired cooking result.

<FIG> is a schematic drawing of a non-transitory machine-readable medium <NUM> (not according to the present invention) for implementing various embodiments described herein. The machine-readable medium <NUM> stores instructions <NUM> readable and executable by processing circuitry <NUM> which, when executed by the processing circuitry <NUM>, cause the processing circuitry <NUM> to implement the method of any of the embodiments described herein (e.g., cooking methods <NUM>, <NUM>, <NUM>, <NUM>, <NUM> and/or flowchart <NUM>). The machine-readable medium <NUM> and/or the processing circuitry <NUM> may be implemented by any of the controller <NUM>, cloud computing service <NUM>, user equipment <NUM> and/or controller <NUM> of <FIG> or <FIG>.

<FIG> is a schematic drawing of apparatus <NUM> for implementing various embodiments described herein. The apparatus <NUM> is implemented by the controller <NUM> of the apparatus according to appended claim <NUM>.

The apparatus <NUM> comprises processing circuitry <NUM>. The processing circuitry <NUM> is configured to communicate with an interface <NUM>. The interface <NUM> may be any interface (wireless or wired) implementing a communications protocol to facilitate exchange of data with other devices such as another part of the cooking ecosystem <NUM>.

The apparatus <NUM> further comprises a non-transitory machine-readable medium <NUM> storing instructions <NUM> which, when executed by the processing circuitry <NUM>, cause the processing circuitry <NUM> to implement the method of at least appended claim <NUM>.

Claim 1:
A computer-implemented cooking method (<NUM>), comprising:
receiving (<NUM>) first image data corresponding to a view associated with a solid food item at a first time of a cooking process implemented by a cooking apparatus;
receiving (<NUM>) second image data corresponding to a view associated with the solid food item at a second time of the cooking process;
comparing (<NUM>) the first image data with the second image data to detect (<NUM>) a geometric change indicative of liquid loss from the solid food item between the first time and the second time as a result of the cooking process, wherein the geometric change is detected by identifying a change in position and/or size of at least part of the solid food item between the first and second image data;
identifying (<NUM>) a transition from a first phase to a second phase of the cooking process where liquid loss from the solid food item as a result of the cooking process occurs at a higher rate during the second phase than the first phase by comparing the identified change in position and/or size with a threshold geometrical change value indicative of the transition, wherein the transition from the first phase to the second phase occurs when, during the cooking process, a structure of the solid food item undergoes a physical change as a result of the cooking process such that liquid is released from the structure; and
in response to the identified change in position and/or size crossing the threshold geometrical change value, causing (<NUM>) the cooking apparatus to modify the cooking process by reducing a cooking temperature of the cooking apparatus.