Patent Description:
Connected cooking appliances, e.g. connected Airfryers, are becoming increasingly common in as part of household kitchens. Currently available connected cooking appliances offer mobile applications (on a Smartphone or a tablet) in order to allow users to search for recipes and perform remote appliance control.

<CIT> describes a method comprising: receiving user-specific health data that can be assigned to a user profile; generation of an evaluation of health data for the user profile by a computing unit; determining a recipe suggestion based on the evaluation of health data; and outputting of the recipe suggestion at a user interface of a kitchen appliance. <CIT> describes a cooking management method for planning a cooking process of a kitchen machine, comprising: determining an entertainment program of an entertainment apparatus; and determining a planning interval in which at least one user of the kitchen machine is connected to the entertainment machine.

The invention is as defined in and by the appended claims. By introducing connectivity into cooking appliances and systems, various user experience problems have surfaced. There are a number of shortcomings associated with currently available connected cooking appliance systems, some of which relating to access to control and cooking status related notifications. For example, mobile applications that are offered together with cooking appliances can provide cooking related feedback and/or instructions, but they are provided regardless of where the user is located or what kind of activity they are performing. For example, users may find it strange or awkward to have to monitor the cooking status of a cooking appliance through their smartphones when they are next to the cooking appliance. Another shortcoming relating to currently available connected cooking systems is that notifications and exposure of controls do not take into account the context and/or the location of the user(s).

It is recognised in the present disclosure that once users initiate remote cooking, the way notifications are provided as well as access to controls become more important in order to achieve a good result. It would be advantageous to provide a technique that allows users to perform control of the cooking appliance readily and instantly, as well as to provide users with the feeling of having quick and easy access to controls even when they may not be paying attention during the cooking process or have to walk away from the cooking appliance for periods of time.

According to a first specific aspect, there is provided a computer-implemented method for controlling an apparatus, as defined in claim <NUM>.

In some embodiments, the apparatus may be a cooking appliance, and a current process being performed by the apparatus may be a cooking process or a step in a cooking process.

In some embodiments, determining at least one of the delivery scheme for the notification and the access scheme for providing control access of the apparatus to the at least one user may be based on a set of predefined rules.

In some embodiments, the delivery scheme may further indicate a sequence in which the notification is provided at the one or more destinations for the notification.

In some embodiments, the one or more salience parameters may comprise at least one of: a type of notification, a volume level associated with an audio notification, a font size associated with a textual notification, a vibration level associated with a haptic notification, wherein the type of notification indicates whether the notification is to be provided in at least one of the following manner: audio, visual, textual, and haptic.

In some embodiments, determining at least one of the delivery scheme for the notification and the access scheme for providing control access of the apparatus to the at least one user may be based on whether at least one of the one or more users is interacting with the apparatus. In these embodiments, whether at least one of the one or more users is interacting with the apparatus may be indicated by the data associated with the activity being performed by the one or more users.

In some embodiments where the notification is associated with a required action from at least one of the one or more users, the method may further comprise, subsequent to providing the notification according to the determined delivery scheme determining whether the required action has been performed within an associated time period. In these embodiments, if it is determined that the required action has not been performed within the associated time period, the method may further comprise performing at least one of the following steps: changing a current operation mode of the apparatus, and determining at least one of a delivery scheme for a new notification and a new access scheme for providing control access of the apparatus to the at least one user. In these embodiments, a context of the new notification may be associated with at least one of: a new current process being performed by the apparatus, a new current status of the apparatus, and a new required action by at least one of the one or more users at the apparatus.

In some embodiments where the notification is associated with a required action from at least one of the one or more users, the delivery scheme may further indicate that an adjustment to the at least one of the one or more destinations at which the notification is provided so as to reduce a number of steps required to achieve the required action. The adjustment may comprise at least one of: providing a shortcut associated with the required action at a user interface of a respective destination, and promoting a control element associated with the required action at a user interface of a respective destination.

In some embodiments, determining at least one of the delivery scheme for the notification and the access scheme for providing control access of the apparatus to the at least one user may be based on respective proximities of more than one user with the apparatus, and the delivery scheme may indicate respective sets of salience parameters for delivering the notification at more than one destinations.

In some embodiments, the data associated with the current location of the one or more users may comprise at least one of: data from a satellite-based radionavigation system, signals from one or more accelerometers at the one or more external devices, signals from one or more passive infrared sensors at the apparatus and/or the one or external devices, signals from one or more audio sensors at the apparatus and/or the one or more external devices, signals from one or more ultrasound sensors at the apparatus and/or the one or more external devices, radio frequency signals, Bluetooth signals, signals associated with a wireless access point, and signals from one or more computer vision based location sensors.

In some embodiments, the data associated with an activity being performed by the one or more users may comprise at least one of: signals from one or more image sensors, signals from one or more motion sensors, signals from an eye tracking operation based on image signals associated with the one or more users, signals from a facial recognition operation based on images signals associated with the one or more users, signals from one or more audio sensors at the apparatus and/or the one or more external devices.

In some embodiments where the method comprises determining the delivery scheme for the notification, the method may further comprise providing the notification according to the determined delivery scheme.

According to a second aspect, there is provided a computer program as defined in claim <NUM>.

According to a third aspect, there is provided an apparatus as defined in claim <NUM>.

According to the proposed method and apparatus of the present disclosure, apparatuses (such as cooking appliances) can operate with an adaptive behaviour which improves the usability and user-friendliness of the apparatuses as well as the mobile applications that are used to control or monitor the apparatuses. Moreover, according to some embodiments of the present disclosure, the adaptive behaviour would reduce the number of steps required for users to instruct certain operations or achieve certain desired effects, and therefore offer improvements in terms of reducing the amount of computer resources required.

As noted above, there is provided an improved apparatus and a method of controlling the same which address the existing problems. It will be appreciated that although the description below includes examples pertaining to cooking appliances, the described method and apparatus are applicable to other (non-cooking related) types of apparatuses, such as baby monitors. It will be appreciated how determined delivery scheme(s) and/or access scheme(s) can be applied to the operation of non-cooking related apparatuses.

<FIG> illustrates a computer-implemented method for controlling an apparatus, according to an embodiment. For example, the method may be used for controlling the apparatus <NUM> described below with reference to <FIG>. In some embodiments, the apparatus may be a cooking appliance, such as an Airfryer, a soup maker, a coffee machine, a steam cooker, etc..

The illustrated method can generally be performed by or under the control of a processor at the apparatus, and/or by a combination of a communication unit, an acquiring unit, and a determining unit at the apparatus, as will be explained in more detail below with reference to <FIG>. The apparatus is wirelessly connected to one or more external devices, such as a smartphone, a tablet, a smart watch, or any other similar devices, etc. In some embodiments, the illustrated method may be implemented as an application software at the apparatus, and in some embodiments the illustrated method may be further implemented as an application software at the one or more external devices. In some embodiments, the illustrated method may be implemented on a cloud infrastructure.

With reference to <FIG>, at block <NUM>, data associated with a current location of each of one or more users and data associated with an activity being performed by each of the one or more users is acquired.

The data associated with the current location of the one or more users may comprise at least one of: data from a satellite-based radionavigation system, signals from one or more accelerometers at the one or more external devices, signals from one or more passive infrared sensors at the apparatus and/or the one or external devices, signals from one or more audio sensors at the apparatus and/or the one or more external devices, signals from one or more ultrasound sensors at the apparatus and/or the one or more external devices, radio frequency signals, Bluetooth signals, signals associated with a wireless access point (e.g. a Wi-Fi access point, if the respective user is remote), and signals from one or more computer vision based location sensors.

The data associated with an activity being performed by the one or more users may comprise at least one of: signals from one or more image sensors, signals from one or more motion sensors, signals from an eye tracking operation based on image signals associated with the one or more users, signals from a facial recognition operation based on images signals associated with the one or more users, signals from one or more audio sensors at the apparatus and/or the one or more external devices. The data associated with an activity being performed by the one or more users may indicate whether a respective user is interacting with the apparatus.

Subsequently, at block <NUM>, at least one of a delivery scheme for delivering a notification at one or more destinations and an access scheme for providing control access of the apparatus to at least one of the one or more users is determined.

The delivery scheme indicates one or more destinations at which the notification is provided (e.g. the apparatus itself and/or the one or more external devices), and one or more salience parameters for delivering the notification at the one or more destinations. The one or more salience parameters are indicative of a salience level for the notification. In some embodiments, the delivery scheme may further indicate a sequence in which the notification is provided at the one or more destinations for the notification.

The access scheme indicates one or more destinations at which control for the apparatus is enabled.

In some embodiments, the one or more salience parameters may comprise at least one of: a type of notification (which indicates whether the notification is to be provided in at least one of the following manner: audio, visual, textual, and haptic), a volume level (e.g. "high", "medium", "low") associated with an audio notification, a font size (e.g. "big", "medium", "small") associated with a textual notification, a vibration level (e.g. "high", "medium", "low") associated with a haptic notification. For example, some notifications may be provided in audio only or vibration only, and some notifications may be provided in both audio and visual formats. These are indicated by the one or more salience parameters. Such salience parameters are indicative of the "salience level", which refers to the importance or prominence with which the notification is to be provided at the one or more destinations.

The determination at block <NUM> is based on the data associated with the current location of each of the one or more users, the data associated with the activity being performed by each of the one or more users, and a context of the notification. The context of the notification is associated with at least one of: a current process being performed by the apparatus, a current status of the apparatus (e.g. a temperature inside the apparatus, a temperature of a food item inside the apparatus, whether the apparatus on/off, online/offline, or in a sleep mode, etc.), and a required action by at least one of the one or more users at the apparatus. In embodiments where the apparatus is a cooking appliance, the current process being performed by the apparatus may be a cooking process or a step in a cooking process (e.g. cutting, frying, steaming, grilling, etc.). The notification may be a target temperature of a food item (e.g. "steak target temperature: <NUM>"), and/or a current temperature of a food item (e.g. "current temperature: <NUM>), and/or a recommended action to be performed by a user (e.g. "turn the steak over"), and/or a status of the apparatus (e.g. "offline for <NUM>").

For example, based on proximity (extracted from data associated with the location of a user) it would be possible to predict how soon the user can reach the apparatus. For instance, if the user is <NUM> metres away from the cooking appliance, the time for them to turn the steak over is about <NUM> seconds, and the time for them to reach their smartphone and pause the cooking process at the cooking appliance via a mobile application is around <NUM> seconds. If the user is in the garden, the time for them to interact with the cooking appliance may be around <NUM> seconds, and the time for them to reach their smartphone and pause the cooking process at the cooking appliance via a mobile application is around <NUM> seconds. Accordingly, if it is determined from location data that the user is near the cooking appliance, the delivery scheme may be determined such that the notification is only provided at the cooking appliance (and not any of the external devices) and the access scheme may be determined such that control is only enabled at the cooking appliance itself. If it is determined from location data the user is in the garden, the delivery scheme may be determined such that the notification is provided at the smartphone of the user (via the mobile application) and the access scheme may be determined such that the user is allowed to turn off the cooking appliance via the smartphone.

In some embodiments where the notification is associated with a required action from at least one of the one or more users, the delivery scheme may further indicate that an adjustment to the at least one of the one or more destinations at which the notification is provided so as to reduce a number of steps required to achieve the required action. The adjustment may comprise at least one of: providing a shortcut associated with the required action at a user interface of a respective destination, and promoting a control element associated with the required action at a user interface of a respective destination. For example, if a required action of the user is to stop a cooking process, the determined delivery scheme for the respective notification may indicate that the "stop" button element provided at a user interface of the apparatus is to be highlighted by way of surrounding LED lights.

As another example, if a required action of the user is to adjust a timer of a cooking process (e.g. frying) at the apparatus, the determined delivery scheme may indicate that a timer control/adjustment screen is to be automatically provided via a user interface of the apparatus, instead of requiring the user to perform default prerequisite step(s) in order to initiate the timer control/adjustment screen (e.g. touching the button in a general menu at the user interface for "timer control/adjustment"). Accordingly, instead of having to perform <NUM> "clicks" in order to adjust the timer, the user only has to perform <NUM> "click". In some embodiments, the determined delivery scheme may further indicate that feedback such as a suggested adjustment to the timer (e.g. "add <NUM> minutes to timer") is to be provided via the user interface of the apparatus.

By providing such shortcuts and/or promoting certain features, the technique proposed in the present disclosure can allow users to perform required actions at the apparatus in an efficient way so as to achieve the desired effects, thus saving time and reducing the amount of interactions needed between users and the apparatus, as well as eliminating the need to display and process unnecessary data/signals in order for the user to perform the required action, thus reducing the amount of computer resources required for performing certain actions at the apparatus itself.

In some embodiments, determining at least one of the delivery scheme for the notification and the access scheme for providing control access of the apparatus to the at least one user at block <NUM> may be based on a set of predefined rules. Table <NUM> and Table <NUM> below outline a number of exemplary predefined rules that can be used for controlling an Airfryer and/or a soup maker.

In some embodiments, determining at least one of the delivery scheme for the notification and the access scheme for providing control access of the apparatus to the at least one user at block <NUM> may be based on whether at least one of the one or more users is interacting with the apparatus (e.g. physically interacting with the apparatus so that the user is paying attention to the apparatus, or interacting with the apparatus via a mobile application on one of the external devices, such as a smartphone). In these embodiments, whether at least one of the one or more users is interacting with the apparatus may be indicated by the data associated with the activity being performed by the one or more users acquired at block <NUM>.

In some embodiments, determining the delivery scheme for the notification and the access scheme for providing control access of the apparatus to the at least one user at block <NUM> may be based on respective proximities of more than one user with the apparatus (or with the one or more external devices). In these embodiments, the delivery scheme may indicate respective sets of salience parameters for delivering the notification at more than one destinations.

For example, the salience parameters for delivering the notification to a first user, who is closer to the apparatus, may indicate higher salience (e.g. audio notification with a high volume) for the respective notification (e.g., a higher "salience level"); the salience parameters for delivering the notification to a second user, who is not close to the apparatus (e.g. in the garden), may indicate lower salience for the respective notification (e.g. only to be provided as a text alert via a mobile application of an external device worn by the second user) (e.g., a lower "salience level"). Although not shown in <FIG>, in some embodiments where the method comprises determining the delivery scheme for the notification, the method may further comprise providing the notification according to the determined delivery scheme.

In embodiments where the notification is associated with a required action from at least one of the one or more users and where the method further comprises providing the notification according to the delivery scheme determined at block <NUM>, subsequent to providing the notification, the method may comprise determining whether the required action has been performed within an associated time period. The determination of whether the required action has been performed within the associated time period may be based on, for example, signals from image sensors (or other types of sensors) at the apparatus. For example, a camera can be provided at the apparatus (which is a cooking appliance in this example) to capture image data of the food item inside the apparatus before (or at the point of) providing the notification, and after the associated time period has expired. The image data can then be compared to determine whether the food item in the cooking appliance has been moved/turned over by the user (in the case where the required action is "turn the food item over") or whether a new ingredient has been introduced into the cooking appliance (in the case where the required action is "add ingredient"), based on whether there are any changes in the image data. Specifically, a change in orientation of the food item or a change in volume of the food item can be determined based on comparison of the image data (e.g. by analysing pixels).

If it is determined that the required action has not been performed within the associated time period, the method may further comprise changing a current operation mode of the apparatus (e.g. switching off the apparatus, stopping a cooking process performed by the apparatus, etc.), and/or determining at least one of a delivery scheme for a new notification and a new access scheme for providing control access of the apparatus to the at least one user. The context of the new notification may be associated with at least one of: a new current process being performed by the apparatus, a new current status of the apparatus, and a new required action by at least one of the one or more users at the apparatus.

For example, if it is determined that the initial required action from the user to "turn the steak over" is not performed in the associated time period, the context of the new notification may be, for example, to request the user to stop the cooking process at the apparatus. The delivery scheme for the new notification may thus be to indicate that the "stop" button (visual element) at the apparatus and/or provided via a mobile application at an external device is to be promoted.

<FIG> is a block diagram of an apparatus according to an embodiment. The apparatus <NUM> comprises a communication unit <NUM>, an acquiring unit <NUM>, and a determining unit <NUM>. In some embodiments, the apparatus <NUM> may be a cooking appliance, such as an Airfryer, a soup maker, a coffee machine, a steam cooker, etc..

The communication unit <NUM> is configured to wirelessly connect to one or more external devices. The one or more external devices may be a smartphone, a tablet, a smart watch, or any other similar devices, etc..

The acquiring unit <NUM> is configured to acquire data associated with a current location of each of one or more users and data associated with an activity being performed by each of the one or more users.

The data associated with the current location of the one or more users may comprise at least one of: data from a satellite-based radionavigation system, signals from one or more accelerometers at the one or more external devices, signals from one or more passive infrared sensors at the apparatus and/or the one or external devices, signals from one or more audio sensors at the apparatus and/or the one or more external devices, signals from one or more ultrasound sensors at the apparatus and/or the one or more external devices, radio frequency signals, Bluetooth signals, signals associated with a wireless access point, and signals from one or more computer vision based location sensors.

The data associated with an activity being performed by the one or more users may comprise at least one of: signals from one or more image sensors, signals from one or more motion sensors, signals from an eye tracking operation based on image signals associated with the one or more users, signals from a facial recognition operation based on images signals associated with the one or more users, signals from one or more audio sensors at the apparatus and/or the one or more external devices.

The determining unit <NUM> is configured to determine at least one of a delivery scheme for delivering a notification at one or more destinations and an access scheme for providing control access of the apparatus to at least one of the one or more users, based on the data associated with the current location of each of the one or more users, the data associated with the activity being performed by each of the one or more users, and a context of the notification.

The context of the notification is associated with at least one of: a current process being performed by the apparatus, a current status of the apparatus, and a required action by at least one of the one or more users at the apparatus. For example, if the apparatus <NUM> is a cooking appliance, the current process being performed by the apparatus is a cooking process or a step in a cooking process. The delivery scheme indicates one or more destinations at which the notification is provided (e.g. the apparatus itself and/or the one or more external devices), and one or more salience parameters for delivering the notification at the one or more destinations. The one or more salience parameters are indicative of a salience level for the notification. The one or more salience parameters may comprise at least one of: a type of notification, a volume level associated with an audio notification, a font size associated with a textual notification, a vibration level associated with a haptic notification, wherein the type of notification indicates whether the notification is to be provided in at least one of the following manner: audio, visual, textual, and haptic.

In some embodiments, the delivery scheme may further indicate a sequence in which the notification is provided at the one or more destinations for the notification. In embodiments where the notification is associated with a required action from at least one of the one or more users, the delivery scheme may further indicate that an adjustment to the at least one of the one or more destinations at which the notification is provided so as to reduce a number of steps required to achieve the required action. The adjustment may comprise at least one of: providing a shortcut associated with the required action at a user interface of a respective destination, and promoting a control element associated with the required action at a user interface of a respective destination. The access scheme indicates one or more destinations at which control for the apparatus is enabled.

In some embodiments, the determining unit <NUM> may be configured to determine the at least one of the delivery scheme for the notification and the access scheme for providing control access of the apparatus to the at least one user based on a set of predefined rules.

In some embodiments, the determining unit <NUM> may be configured to determine the at least one of the delivery scheme for the notification and the access scheme for providing control access of the apparatus to the at least one user based on respective proximities of more than one user with the apparatus <NUM> (or with the one or more external devices). In these embodiments, the delivery scheme may indicate respective sets of salience parameters for delivering the notification at more than one destinations.

In some embodiments, the determining unit <NUM> may be configured to determine the at least one of the delivery scheme for the notification and the access scheme for providing control access of the apparatus to the at least one user based on whether at least one of the one or more users is interacting with the apparatus (e.g. physically interacting with the apparatus so that the user is paying attention to the apparatus, or interacting with the apparatus via a mobile application on one of the external devices, such as a smartphone). In these embodiments, whether at least one of the one or more users is interacting with the apparatus may be indicated by the data associated with the activity being performed by the one or more users acquired by the acquiring unit <NUM>.

In some embodiments where the determining unit <NUM> is configured to determine the delivery scheme for the notification, the communication unit <NUM> may be configured to provide the notification according to the determined delivery scheme.

In embodiments where the notification is associated with a required action from at least one of the one or more users, and the communication unit <NUM> is configured to provide the notification according to the determined delivery scheme, the determining unit <NUM> may be further configured to, subsequent to providing the notification according to the determined delivery scheme, determine whether the required action has been performed within an associated time period. Furthermore, if it is determined that the required action has not been performed within the associated time period, the apparatus <NUM> may be configured to change a current operation mode (e.g. switch off, or pausing/stopping a current cooking process), and/or the determining unit <NUM> may be configured to further determine at least one of a delivery scheme for a new notification and a new access scheme for providing control access of the apparatus to the at least one user. The context of the new notification is associated with at least one of: a new current process being performed by the apparatus, a new current status of the apparatus, and a new required action by at least one of the one or more users at the apparatus.

It will be appreciated that <FIG> only shows the components required to illustrate an aspect of the apparatus <NUM> and, in a practical implementation, the apparatus <NUM> may comprise alternative or additional components to those shown. For example, in some embodiments the apparatus <NUM> may further comprise a power source, and/or a user interface, and/or a memory.

The user interface may be for use in providing a user of the apparatus <NUM> with information resulting from the technique described herein. Alternatively or in addition, the user interface may be configured to receive a user input. For example, the user interface may allow a user of the apparatus <NUM> to manually enter instructions, data, or information. The user interface may be any user interface that enables the rendering (or output or display) of information to a user of the apparatus <NUM>. Alternatively or in addition, the user interface may be any user interface that enables a user of the apparatus <NUM> to provide a user input, interact with and/or control the apparatus <NUM>. For example, the user interface may comprise one or more switches, one or more buttons, a keypad, a keyboard, a touch screen or an application (for example, on a tablet or smartphone), a display screen, a graphical user interface (GUI) or other visual rendering component, one or more speakers, one or more microphones or any other audio component, one or more lights, a component for providing tactile feedback (e.g. a vibration function), or any other user interface, or combination of user interfaces.

As used herein, the expression "at least one of' followed by a recited set of elements or features refers to any number or combination of the recited set of elements or features. For example, where the expression "at least one of' is used, this may refer to "one of' or "a plurality of' the recited set of elements or features. In other words, for the recited set of elements/features: A, B and C, the expression "at least one of A, B and C" means "A, B, and/or C", and that it suffices if only B is present, for example.

Claim 1:
A computer-implemented method for controlling an apparatus, wherein the apparatus is wirelessly connected to one or more external devices, the method comprising:
acquiring (<NUM>) data associated with a current location of each of one or more users and data associated with an activity being performed by each of the one or more users;
the method being characterized by the following steps:
determining (<NUM>) at least one of a delivery scheme for delivering a notification at one or more destinations and an access scheme for providing control access of the apparatus to at least one of the one or more users, wherein the determining is based on:
the data associated with the current location of each of the one or more users;
the data associated with the activity being performed by each of the one or more users; and
a context of the notification, wherein the context of the notification is associated with at least one of: a current process being performed by the apparatus, a current status of the apparatus, and a required action by at least one of the one or more users at the apparatus,
wherein the delivery scheme indicates one or more salience parameters for delivering the notification at the one or more destinations, wherein the one or more salience parameters are indicative of an importance or prominence with which the notification is to be provided at the one or more destinations, and
wherein the access scheme indicates the one or more destinations at which control for the apparatus is enabled.