Patent Description:
Internet of things (IoT) refers to a network of devices that are connected to one another via the Internet. An loT environment comprises multiple objects including loT devices, non-loT devices, and the like. These objects are a part of a routine of a user. The user uses the objects to perform various activities. For example, the loT environment is a smart home including the objects, such as, smart Televisions (TVs), coffee maker, mobile phone, iron, and the like. It is most likely that the user occasionally experiences a memory loss or notion regarding an intended activity of the user, especially when the intended activity is interrupted by another important or urgent activity. For example, the user may want to switch off the iron. However, the user may be interrupted by ringing of a doorbell. In such case, it may be likely that the user may miss performing the intended activity i.e., switching off the iron. This loss of track of activities may cause major problems in daily life. <CIT> describes in response to determining that an activity has been postponed (e.g., interrupted or deferred), a computer system stores a record indicating that the activity is postponed.

There are some systems of the related art for reminding the user to perform the activities. These systems of the related art include calendar reminder systems, To-do list systems, and other monitoring systems. In these systems of the related art, the user has to pre-log the activities, such as provide a to-do list to get the reminders to perform the activities. In some other systems of the related art, the user has to manually set the reminders for reminding to perform certain activities. In addition, these reminders are not dynamic in nature. In addition, the systems of the related art are limited to loT activities and do not consider non-loT activities. Further, in the systems of the related art, the reminders for the activities are limited to pre-logged activities. These systems do not remind the user to perform any activities other than the pre-logged activities.

Accordingly, an aspect of the disclosure is to provide a method and a reminder generation IoT device for generating activity reminder in the loT environment.

In the disclosure, the word "exemplary" is used herein to mean "serving as an example, instance, or illustration. " Any embodiment or implementation of the subject matter described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

While the disclosure is susceptible to various modifications and alternative forms, specific embodiment thereof has been shown by way of example in the drawings and will be described in detail below. It should be understood, however that it is not intended to limit the disclosure to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the scope of the disclosure.

The terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a setup, device or method that comprises a list of components or steps does not include only those components or steps but may include other components or steps not expressly listed or inherent to such setup or device or method. In other words, one or more elements in a system or apparatus proceeded by "comprises. a" does not, without more constraints, preclude the existence of other elements or additional elements in the system or apparatus.

Embodiments of the disclosure relate to a method and a reminder generation Internet of things (IoT) device for generating an activity reminder in an loT environment. The loT environment may comprise multiple objects. A user associated with the loT environment may intend to perform an activity associated with an object. A movement of the user towards the object is identified. Further, an intended activity of the user, associated with the object is predicted. Consider that before performing the intended activity, the user may get interrupted. In such condition, an occurrence of one or more interrupts is determined. The user may either complete the interrupted activity or may leave it incomplete due to the occurrence of the one or more interrupts. In case the user moving away from the object subsequent to the occurrence of the one or more interrupts is detected, an activity reminder is generated to the user. The activity reminder is generated for performing the intended activity when the user does not move towards the object for a pre-determined time period. As a result, the disclosure aids in reminding the user to perform the intended activity in the loT environment. Further, these reminders are dynamic in nature. The disclosure avoids the need of pre-logging the activities or manually setting the reminders. In addition, the disclosure considers IoT activities and non-loT activities. Further, the disclosure considers interrupts caused by loT events and non-loT events.

<FIG> illustrates an environment for generating an activity reminder in an IoT environment according to an embodiment of the disclosure.

Referring to <FIG>, an environment <NUM> is an IoT environment <NUM>. The loT environment <NUM> comprises a user <NUM>, an object (<NUM><NUM>, <NUM><NUM>,. <NUM>N) (collectively referred as plurality of objects <NUM>), an Ultra-wideband (UWB) sensor (<NUM><NUM>, <NUM><NUM>,. <NUM>N) (collectively referred as one or more UWB sensors <NUM>) corresponding to each of the plurality of objects <NUM>. Any object in the plurality of objects <NUM> may be implemented for reminder generation. For instance, referring to <FIG>, the object <NUM><NUM> is configured as the reminder generation IoT device <NUM><NUM>. For example, the object may be a TV. The plurality of objects may include loT objects/devices or non-loT objects. For example, in any smart home loT environment, the loT devices may include a smart television (TV), a speaker, a mobile phone, an oven, a refrigerator, and the like. The non-loT objects may include a gas stove, an iron, a food processor, and the like. In another example, the loT environment <NUM> may be a smart office. The loT objects may include a laptop, an air conditioner (AC), a mobile phone, and the like. The non-loT objects may include a comfortable chair, a file cabinet, and the like. The user <NUM> may use the plurality of objects <NUM> to perform activities in the loT environment <NUM>. For example, the object may be a TV. The user <NUM> may perform the activity of watching TV. In another example, the activity may be loading clothes in a washing machine and switching on the washing machine. The IOT environment <NUM> may comprise multiple users performing respective activities. <FIG> illustrates one user <NUM> for illustrative purposes only and should not be considered as limiting.

Referring to <FIG>, at operation <NUM>, the user <NUM> is moving towards the object <NUM><NUM>. Consider, the object <NUM><NUM> may be an AC. The user <NUM> may be interrupted by a beeping sound from a refrigerator, indicating that door of the refrigerator is open. The user <NUM> may forget to operate the AC. In the disclosure, the reminder generation loT device <NUM><NUM> generates the activity reminder to the user <NUM>, indicating to operate the AC. The reminder generation loT device <NUM><NUM> identifies a movement of the user <NUM> towards the object <NUM><NUM> in the loT environment <NUM>. The movement of the user <NUM> may be identified based on data from the one or UWB sensors <NUM> in the loT environment <NUM>. The UWB is a short-range, wireless communication protocol that operates through radio waves and operates at very high frequencies. A UWB sensor can be used to capture highly accurate spatial and directional data. The UWB sensor can discover location of an object and communicate with the object. The one or more UWB sensors may be associated with a device associated with the user <NUM>, the object <NUM><NUM>, other objects from the plurality of objects <NUM>, and the like. Further, the reminder generation loT predicts a first intended activity of the user <NUM> associated with the object <NUM><NUM>, based on one or more factors. For example, the first intended activity may be predicted as change settings of the AC. The user <NUM> may be interrupted before performing the first intended activity. The reminder generation IoT device <NUM><NUM> determines occurrence of one or more interrupts in the loT environment <NUM> before performing the first intended activity associated with the object. For example, at operation <NUM>, the user <NUM> is interrupted by ringing of a doorbell. The reminder generation loT device <NUM><NUM> determines the occurrence of this interrupt. The user <NUM> may either complete the interrupted activity or may leave it incomplete due to the occurrence of the interrupt. The reminder generation loT device <NUM><NUM> detects the movement of the user <NUM> away from the object <NUM><NUM> in case the user <NUM> moves away before performing the first intended activity. For example, at operation <NUM>, the user <NUM> is moving away from the object <NUM><NUM>. In such condition, the reminder generation loT device <NUM><NUM> may determine whether the user <NUM> returns to the object <NUM><NUM> within a pre-determined time period. In case the user <NUM> does not return to the object <NUM><NUM> for the pre-determined time period, the reminder generation IoT device <NUM><NUM> may generate the activity reminder to the user <NUM>, indicating to perform the first intended activity when the user <NUM> does not move towards the object <NUM><NUM> for the pre-determined time period. For example, the reminder generation loT device <NUM><NUM> may generate the reminder after <NUM> secs when the user <NUM> does not move towards the AC. The reminder may be generated in form of a text, audio, and the like. For example, the reminder may be generated and displayed as a text in a smart screen <NUM><NUM>.

<FIG> illustrates a diagram <NUM> of a reminder generation IoT device for generating an activity reminder in an IoT environment according to an embodiment of the disclosure.

Referring to <FIG>, the reminder generation loT device <NUM><NUM> is referred as the reminder generation loT device <NUM> hereafter in the description. The reminder generation IoT device <NUM> may include central processing units <NUM> (also referred as "CPUs" or "one or more processors <NUM>"), Input/ Output (I/O) interface <NUM>, and a memory <NUM>. In some embodiments of the disclosure, the memory <NUM> may be communicatively coupled to the one or more processors <NUM>. The memory <NUM> stores instructions executable by the one or more processors <NUM>. The one or more processors <NUM> may comprise at least one data processor for executing program components for executing user or system-generated requests. The memory <NUM> may be communicatively coupled to the one or more processors <NUM>. The memory <NUM> stores instructions, executable by the one or more processors <NUM>, which, on execution, may cause the one or more processors <NUM> to generate the activity reminder in the loT environment <NUM>. In an embodiment of the disclosure, the memory <NUM> may include one or more modules <NUM> and data <NUM>. The one or more modules <NUM> may be configured to perform the steps of the disclosure using the data <NUM>, to generate the activity reminder in the loT environment <NUM>. In an embodiment of the disclosure, each of the one or more modules <NUM> may be a hardware unit which may be outside the memory <NUM> and coupled with the reminder generation IoT device <NUM>. As used herein, the term modules <NUM> refers to an application specific integrated Circuit (ASIC), an electronic circuit, a field-programmable gate arrays (FPGA), programmable system-on-chip (PSoC), a combinational logic circuit, and/or other suitable components that provide described functionality. The one or more modules <NUM> when configured with the described functionality defined in the disclosure will result in a novel hardware. Further, the I/O interface <NUM> is coupled with the one or more processors <NUM> through which an input signal or/and an output signal is communicated. For example, the reminder generation loT device <NUM> may transmit the generated reminder via the I/O interface <NUM> to the user <NUM>. In an embodiment of the disclosure, the reminder generation IoT device <NUM> may be an IoT device of the loT environment <NUM>. For example, the reminder generation IoT device <NUM> may be a computer, TV, and the like. In another embodiment of the disclosure, the reminder generation IoT device <NUM> may be implemented in a variety of computing systems, such as a laptop computer, a desktop computer, a personal computer (PC), a notebook, a smartphone, a tablet, e-book readers, a server, a network server, a cloud-based server, and the like. In another embodiment of the disclosure, the reminder generation IoT device <NUM> may be remote to the loT environment <NUM> and communicate with the plurality of objects <NUM>, the user <NUM>, and the like in the loT environment <NUM> over a communication network. The reminder generation loT device <NUM> may be implemented in a cloud environment.

In one implementation, the modules <NUM> may include, for example, a movement identification module <NUM>, an activity prediction module <NUM>, an interrupt detection module <NUM>, a movement change detection module <NUM>, a reminder generation module <NUM>, and other modules <NUM>. It will be appreciated that such aforementioned modules <NUM> may be represented as a single module or a combination of different modules. In one implementation, the data <NUM> may include, for example, movement identification data <NUM>, activity prediction data <NUM>, interrupt data <NUM>, movement change detection data <NUM>, reminder generation data <NUM>, and other data <NUM>.

In an embodiment of the disclosure, the movement identification module <NUM> may be configured to identify a movement of the user <NUM> towards the object <NUM><NUM> in the loT environment <NUM>. Firstly, the movement identification module <NUM> may be configured to detect the user <NUM>. The user <NUM> may be detected using a UWB sensor associated with the user <NUM>, the object <NUM><NUM>, other objects from the plurality of objects <NUM>, and the like. For example, the UWB sensor associated with the user <NUM> may transmit the radio signals. The radio signals may transmit a unique identification (ID) associated with the user <NUM>. In another example, a radio signal may be transmitted from the object <NUM><NUM> towards the user <NUM>. The radio signal reflected at the object <NUM><NUM> may be used to detect the user <NUM>. A person skilled in the art will appreciate that any known methods other than the above-mentioned methods may be used to detect the user <NUM>. Further, the movement identification module <NUM> may be configured to identify the movement of the user <NUM> towards the object <NUM><NUM> in the loT environment <NUM>. The object <NUM><NUM> may be an loT device, such as, a smart TV or a non-loT device, such as an iron. The movement of the user <NUM> towards the object <NUM><NUM> may be identified based on data from the one or more UWB sensors <NUM> in the loT environment <NUM>. The data from the one or more UWB sensors <NUM> may comprise a distance between the user <NUM> and the object <NUM><NUM>, signal power associated with a signal reflected from a device associated with the user (<NUM>), the object, and the other objects, angle of arrival of the signal, time difference of arrival of the signal, and the like.

<FIG>, and <FIG> show illustrations for generating an activity reminder in an IoT environment according to various embodiments of the disclosure.

Referring to example <NUM> of <FIG>, at operation <NUM>, the user <NUM> is moving towards a thermostat <NUM>. The thermostat may be associated with a UWB sensor. A signal may be transmitted from the UWB sensor towards the user <NUM>. The signal reflected from the user <NUM> is associated with a signal power. In an embodiment of the disclosure, outliers and interference in the reflected signal may be normalized to avoid incorrect user movement identification. The signal power may increase when the user <NUM> is moving towards the thermostat <NUM>. Further, the distance between the user <NUM> and the object <NUM><NUM> may decrease continuously when the user <NUM> is moving towards the thermostat <NUM>. Hence, the movement identification module <NUM> may identify the movement of the user <NUM> towards the thermostat <NUM> based on the data from the one or more UWB sensors <NUM>.

In another embodiment of the disclosure, the movement identification module <NUM> may identify the movement of the user <NUM> based on an object map. The object map may indicate locations of the plurality of objects <NUM> in the loT environment <NUM>.

Referring to <FIG>, it shows an object map <NUM>. The object map indicates locations of the plurality of objects <NUM> in the IOT environment <NUM>, such as, sofa, thermostat, AC, TV, and the like. The object map may be dynamically updated at pre-defined time intervals since locations of the plurality of objects <NUM> may change in a period of time. The object map may be updated based on the data from the one or more UWB sensors <NUM>, data from an image capturing unit, such as an loT camera, and the like. In an example, the thermostat <NUM> may not be associated with an UWB sensor.

Referring to <FIG>, a movement identification module <NUM> may receive the data from a UWB sensor associated with an AC as shown in <FIG>. A signal received from the UWB sensor associated with the AC may have a low signal power and the distance between the AC and the user <NUM> may be increasing. Further, the data from UWB sensors associated with other objects, such as TV, smart doorbell, and the like may also be received. The data from multiple objects may be consolidated. The movement identification module <NUM> may identity the movement of the user <NUM> towards the thermostat <NUM> based on an object map <NUM> and consolidated data from the multiple objects. Further, data related to identification of the movement of the user <NUM>, the object map <NUM> and consolidated data may be provided to a neural network. For example, the neural network may be a deep Learning Long Short-Term Memory (LSTM) network. The neural network may predict the user movement towards the <NUM><NUM>. The movement identification module <NUM> may receive data related to the prediction from the neural network and may identify the movement of the user <NUM>. The data related to the identification of the movement and the object map <NUM> may be stored as the movement identification data <NUM> in the memory <NUM>.

In an embodiment of the disclosure, the activity prediction module <NUM> may be configured to receive the movement identification data <NUM> from the movement identification module <NUM>. The activity prediction module <NUM> may be configured to predict the first intended activity of the user <NUM>, associated with the object <NUM><NUM> in the loT environment <NUM>. The first intended activity may be predicted by an activity prediction model based on one or more factors. The activity prediction model may be associated with the activity prediction module <NUM>. The activity prediction model may be the neural network (stated in paragraph <NUM>) or any other neural network. For example, the activity prediction model may be reinforcement learning (RL) based deep learning model. A person skilled in the art will appreciate that any known neural networks other than the above-mentioned neural networks may be used to predict the first intended activity of the user <NUM>, based on the one or more factors. The one or more factors may comprise past interactions of the user <NUM> with the object <NUM><NUM>, a current state of the object <NUM><NUM>, and the like. The activity prediction model may learn user activity patterns over-time based on the past interactions of the user <NUM>. For example, the activity prediction model may predict the user is controlling AC temperature, when the movement of the user <NUM> towards the AC is identified. The activity prediction model may predict the first intended activity based on the current state of the object <NUM><NUM>. For example, the AC may be in a switched-on state when the user <NUM> wakes up. The user may move towards the AC. The activity prediction model may predict the first intended activity, based on interactions of the user <NUM> as switching off the AC. Referring again to the example <NUM> of <FIG>, the activity prediction module <NUM> may predict the first intended activity as adjusting the temperature in the thermostat <NUM>. The predicted first intended activity may be stored as the activity prediction data <NUM> in the memory <NUM>.

In an embodiment of the disclosure, the interrupt detection module <NUM> may be configured to receive the activity prediction data <NUM> from the activity prediction module <NUM>. The interrupt detection module <NUM> may be configured to determine an occurrence of the one or more interrupts in the loT environment <NUM> before performing the first intended activity associated with the object. Each of the one or more interrupts may be caused by the loT event or the non-loT event. The interrupt detection module <NUM> may determine the occurrence of the one or more interrupts based on two conditions. Firstly, the interrupt detection module <NUM> may determine the occurrence of the interrupt based on one or more parameters associated with the plurality of objects <NUM> in the loT environment <NUM>. The one or more parameters may include, but not limited to, a state parameter, an audio parameter, a video parameter, and the like. For example, the state of a smart doorbell may change from an OFF state to an ON state when a person is at a door in a smart home. The ringing of the smart doorbell may cause an interrupt when the user <NUM> is intending to perform the first intended activity of switching off the TV. This interrupt is caused by the IoT event, since the interrupt is caused by the smart doorbell which is an IoT device. Secondly, the interrupt detection module <NUM> may determine a change in direction of the user <NUM> away from the object <NUM><NUM> before performing the first intended activity. When the two conditions are satisfied, the interrupt detection module <NUM> determines the occurrence of the one or more interrupts. In the above stated example, the user <NUM> may move away from the TV and attend to the person. The interrupt detection module <NUM> may determine the occurrence of the one or more interrupts based on the state of the smart doorbell and the change in direction of the user <NUM> away from the TV. In another embodiment of the disclosure, consider that the user <NUM> may not immediately attend to the person. Instead, the user <NUM> may switch off the TV and then attend to the person. The interrupt detection module <NUM> may determine that the change in direction of the user <NUM> is not sudden and reaction time is more than a pre-defined value (for example, <NUM> seconds). In such case, the interrupt detection module <NUM> may determine that the loT event is not an interrupt. In another example, a glass cup may fall from a table and break. A speaker nearby the table may capture sound from breaking of the glass cup. The breaking of the glass cup may cause an interrupt when the user <NUM> is intending to perform the first intended activity. The interrupt detection module <NUM> may determine a change in direction of the user <NUM>. The interrupt detection module <NUM> may determine the occurrence of the interrupt based on the audio parameter associated with the breaking of the glass cup and change in direction of the user <NUM>. This interrupt is caused by the non-loT event. Referring again to the example <NUM> of <FIG>, at operation <NUM>, a microwave oven <NUM> may provide a notification that food is heated. The interrupt detection module <NUM> may determine the occurrence of the interrupt based on a completion state associated with the microwave oven <NUM> and the change in direction of the user <NUM>. In an example, the first intended activity of the user <NUM> may be switching off the TV. A first interrupt may be caused by breaking of the glass cup. A second interrupt mat be caused by crying of a kid. The first interrupt and the second interrupt may occur simultaneously. The interrupt detection module <NUM> may determine the audio parameter associated with the breaking of the glass cup and crying of the kid. Further, the interrupt detection module <NUM> may determine the occurrence of the one or more interrupts based on the change in direction of the user <NUM>. In another example, a first interrupt may be a notification from a washing machine indicating that washing of clothes is completed. A second interrupt may be ringing of the smart doorbell. The interrupt detection module <NUM> may determine the state of the washing machine and the smart doorbell to determine the occurrence of the one or more interrupts. Consider that the user <NUM> may not move towards the washing machine immediately to switch off the washing machine, rather the user <NUM> may move towards the door to attend to the person at the door. The interrupt detection module <NUM> may determine the change in direction of the user <NUM> and determine the occurrence of the interrupt i.e., the second interrupt. The data related to determination of the occurrence of the one or more interrupts may be stored as the interrupt data <NUM> in the memory <NUM>.

In an embodiment of the disclosure, the movement change detection module <NUM> may receive the interrupt data <NUM> from the interrupt detection module <NUM>. The movement change detection module <NUM> may be configured to detect the movement of the user <NUM> away from the object <NUM><NUM> subsequent to the occurrence of the one or more interrupts and before performing the first intended activity. The movement of the user <NUM> away from the object <NUM><NUM> may be detected using data from the one or more UWB sensors <NUM> in the IOT environment <NUM>. Referring to the example <NUM> of <FIG>, the user <NUM> moving away from the thermostat <NUM> may be determined based on the data from the UWB sensor associated with the thermostat <NUM>. The data from the UWB sensor may comprise the distance from the thermostat <NUM> and the user <NUM>, the signal power associated with the signal reflected from the user <NUM>, and the like. When the user <NUM> is moving away from the thermostat, the distance from the thermostat <NUM> and the user <NUM> may continuously increase and the signal power may continually decrease. Further, the movement of the user <NUM> away from the object <NUM><NUM> before performing the first intended activity is determined by predicting that the user <NUM> has not completed the first intended activity. The prediction may be performed by the neural network. The prediction may be based on a distance between the user <NUM> and the object, time spent by the user <NUM> near the object <NUM><NUM> before moving away from the object, a current state of the object <NUM><NUM>, and data related to completion of the first intended activity obtained from one or more sources. Referring again to the example <NUM> of <FIG>, the neural network may predict that the user <NUM> has not operated the thermostat <NUM> based on determining that the state of the thermostat or settings on the thermostat are unchanged. Further, the neural network may predict that the user <NUM> has moved away from the thermostat <NUM> when the distance between the thermostat <NUM> and the user <NUM> was large i.e., the user <NUM> had not reached the thermostat <NUM>. In another example, the user <NUM> may be moving near a laundry bag to load clothes in a washing machine. The user <NUM> may be interrupted and move away from the laundry bag. The neural network may predict that the user <NUM> spent less time near the laundry bag (for example, <NUM> or <NUM> seconds) and moved away from the laundry bag. Further, the neural network may obtain the data related to completion of the first intended activity from the one or more sources, such as, IoT camera, a robot camera installed on a vacuum cleaner, and the like. The neural network may determine that the laundry bag is full, and the clothes are not loaded to the washing machine based on an image or a video from the one or more sources. The data related to the movement change may be stored as the movement change detection data <NUM> in the memory <NUM>.

In an embodiment of the disclosure, the reminder generation module <NUM> may be configured to receive the movement change detection data <NUM> from the movement change detection module <NUM>. The reminder generation module <NUM> may be configured to generate the activity reminder to the user <NUM>, indicating to perform the first intended activity when the user <NUM> does not move towards the object <NUM><NUM> for the pre-determined time period. In an embodiment of the disclosure, the pre-determined time period may be <NUM> secs The reminder generation module <NUM> may monitor the movement of the user <NUM> after receiving the movement change detection data <NUM>. The reminder generation module <NUM> may monitor the movement based on the data from the one or more UWB sensors <NUM>. The reminder generation module <NUM> may determine that the user <NUM> has not moved towards the object <NUM><NUM> for the pre-determined time period. Further, the reminder generation module <NUM> may determine whether the first intended activity is completed based on data related to the prediction in the movement change detection data <NUM>. The reminder generation module <NUM> may generate the activity reminder to the user <NUM>, indicating to perform the first intended activity. The reminder generation module <NUM> may generate the activity reminder to the user <NUM> on a nearby object to the user <NUM>. The nearby object may be identified based on at least one of data from the one or more UWB sensors <NUM> and an object map. The nearby object may be an loT device. The reminder generation module <NUM> may identify the nearby object further based on one or more capabilities of the nearby object. The one or more capabilities of the nearby object may comprise audio capability, display capability, and the like. Referring to the example <NUM> in <FIG>, the reminder generation module <NUM> may determine that the user <NUM> has not moved towards the thermostat <NUM> for the pre-determined time period. Thereafter, the reminder generation module <NUM> may determine nearby objects for example, a refrigerator <NUM> and food processor (not shown in <FIG>) based on the data from the UWB sensor associated with the refrigerator <NUM> and the food processor or from the object map. Further, the reminder generation module <NUM> may determine a capability of the refrigerator <NUM> and food processor. The reminder generation module <NUM> may determine that the refrigerator <NUM> comprises display capability. Accordingly, the reminder generation module <NUM> may display the activity reminder on a screen associated with the refrigerator <NUM>. The activity reminder may be in form of a text, audio, and the like. <FIG> shows the activity reminder displayed as a text "Did you miss adjusting temperature I thermostat?". Further, the reminder generation module <NUM> may display options, such as "Yes, get control", "dismiss", and the like. The generated activity reminder may be stored as the reminder generation data <NUM> in the memory <NUM>. For example, the reminder generation data <NUM> may maintain a list of generated activity reminders and pre-defined activity reminders provided by the user <NUM> which may be used by the reminder generation module <NUM> to generate the activity reminder.

The other data <NUM> may store data, including temporary data and temporary files, generated by the one or more modules <NUM> for performing the various functions of the reminder generation IoT device <NUM>. The one or more modules <NUM> may also include the other modules <NUM> to perform various miscellaneous functionalities of the reminder generation IoT device <NUM>. The other data <NUM> may be stored in the memory <NUM>. It will be appreciated that the one or more modules <NUM> may be represented as a single module or a combination of different modules.

<FIG> illustrates a flowchart illustrating method steps for generating an activity reminder in an IoT environment according to an embodiment of the disclosure.

Referring to <FIG>, a method <NUM> may comprise one or more steps. The method <NUM> may be described in the general context of computer executable instructions. Generally, computer executable instructions can include routines, programs, objects, components, data structures, procedures, modules, and functions, which perform particular functions or implement particular abstract data types.

The order in which the method <NUM> is described is not intended to be construed as a limitation, and any number of the described method blocks can be combined in any order to implement the method. Additionally, individual blocks may be deleted from the methods without departing from the scope of the subject matter described herein. Furthermore, the method can be implemented in any suitable hardware, software, firmware, or combination thereof.

<FIG> show illustrations for generating an activity reminder in an loT environment according to various embodiments of the disclosure.

Referring to <FIG>, at operation <NUM>, the reminder generation loT device <NUM> identifies the movement of the user <NUM> towards the object <NUM><NUM> in the IoT environment <NUM>. The reminder generation IoT device <NUM> may be configured to detect the user <NUM>, based on the data from the one or more UWB sensors <NUM>. Further, the reminder generation IoT device <NUM> may identify the movement of the user <NUM> towards the object <NUM><NUM> in the IoT environment <NUM>. The movement of the user <NUM> towards the object <NUM><NUM> may be identified based on data from the one or more UWB sensors <NUM> in the loT environment <NUM>. The reminder generation loT device <NUM> may identify the movement of the user <NUM> further based on the object map. Further, data related to identification of the movement of the user <NUM>, the object map <NUM> and the data from the one or more UWB sensors <NUM> may be provided to the neural network. Referring to example <NUM> of <FIG>, at operation <NUM>, the user <NUM> is moving towards a gas stove <NUM>. The gas stove <NUM> and the user <NUM> may not be associated with a UWB sensor. The reminder generation loT device <NUM> may identify the movement of the user <NUM> towards the gas stove <NUM> based on the data from the one or more UWB sensors <NUM> associated with other objects among the plurality of objects <NUM> and the object map. Referring to <FIG>, an object map <NUM> associated with a kitchen in the smart home is illustrated. The object map shows locations of the plurality of objects <NUM>, such as, the gas stove <NUM>, food processor, refrigerator, oven, smart coffee maker, and the like. The smart coffee maker, the oven, and the refrigerator may be associated with the one or more UWB sensors <NUM>. The data from a UWB sensor associated with the oven may indicate that the user <NUM> is moving away from the oven. Similarly, the data from UWB sensors associated with the smart coffee maker and the refrigerator may indicate that the user <NUM> is moving away from the smart coffee maker and the refrigerator. Further, the user <NUM> may be moving towards the gas stove <NUM> or the food processor. The data from the UWB sensor associated with the refrigerator may indicate that the user <NUM> is not moving towards the food processor. The reminder generation loT device <NUM> may identify that the user <NUM> is moving towards the gas stove <NUM> based on the data from the one or more UWB sensors <NUM> and the object map.

Referring back to <FIG>, at operation <NUM>, the reminder generation IoT device <NUM> predicts the first intended activity of the user <NUM>, associated with the object <NUM><NUM> in the loT environment <NUM>, based on one or more factors. The reminder generation IoT device <NUM> may be configured to predict the first intended activity of the user <NUM>, associated with the object <NUM><NUM> in the loT environment <NUM>. The first intended activity may be predicted by the activity prediction model based on the one or more factors. The activity prediction model may be associated with the reminder generation IoT device <NUM>. The one or more factors may comprise past interactions of the user <NUM> with the object <NUM><NUM>, a current state of the object <NUM><NUM>, and the like. Referring again to the example <NUM> of <FIG>, the reminder generation loT device <NUM> may receive an image from an IoT camera. The reminder generation IoT device <NUM> may determine the as gas stove <NUM> is ON from the image. The reminder generation loT device <NUM> determine that the first intended activity is switching off the gas stove <NUM> based on the past interactions of the user <NUM>.

At operation <NUM>, the reminder generation IoT device <NUM> may be configured to determine an occurrence of the one or more interrupts in the loT environment <NUM> before performing the first intended activity associated with the object. The reminder generation IoT device <NUM> may determine the occurrence of the one or more interrupts based on the one or more parameters associated with the plurality of objects <NUM> in the IoT environment <NUM>. Further, the reminder generation IoT device <NUM> may determine the occurrence of the one or more interrupts based on the change in direction of the user <NUM> away from the object <NUM><NUM> before performing the first intended activity. When the two conditions are satisfied, the reminder generation IoT device <NUM> determines the occurrence of the one or more interrupts. Referring again to the example <NUM> of <FIG>, a kid <NUM> may call his mother (the user <NUM>). A smart speaker <NUM> near the kid may capture the sound and transmit an output proportional to the sound to the reminder generation IoT device <NUM>. The reminder generation IoT device <NUM> may determine the change in direction of the user <NUM>. The reminder generation IoT device <NUM> may determine the occurrence of the one or more interrupts based on the output from the speaker and the change in direction of the user <NUM>.

Referring back to <FIG>, at operation <NUM>, the reminder generation loT device <NUM> may be configured to detecting the movement of the user <NUM> away from the object <NUM><NUM> subsequent to the occurrence of the one or more interrupts and before performing the first intended activity. The movement of the user <NUM> away from the object <NUM><NUM> may be detected using the data from the one or more UWB sensors <NUM> in the IOT environment <NUM>. Further, the movement of the user <NUM> away from the object <NUM><NUM> before performing the first intended activity is determined by predicting that the user <NUM> has not completed the first intended activity. The prediction may be performed by the neural network. The prediction may be based on a distance between the user <NUM> and the object, time spent by the user <NUM> near the object <NUM><NUM> before moving away from the object, a current state of the object <NUM><NUM>, and data related to completion of the first intended activity obtained from one or more sources. Referring again to the example <NUM> of <FIG>, the reminder generation loT device <NUM> may determine the movement of the user <NUM> away from the gas stove <NUM> based on the data from the one or more UWB sensors associated with the other objects. Further, the neural network may predict that the user <NUM> has moved away from the gas stove <NUM> when the distance between the gas stove <NUM> and the user <NUM> was large i.e., the user <NUM> had not reached the gas stove <NUM>. Further, the neural network may determine that the gas stove is in ON state, based on an image or a video from the one or more sources.

Referring back to <FIG>, at operation <NUM>, the reminder generation IoT device <NUM> may generate the activity reminder to the user <NUM>, indicating to perform the first intended activity when the user <NUM> does not move towards the object <NUM><NUM> for the pre-determined time period. The reminder generation loT device <NUM> may monitor the movement of the user <NUM> for the pre-determined time period. The reminder generation IoT device <NUM> may determine that the user <NUM> has not moved towards the object <NUM><NUM> for the pre-determined time period. Further, the reminder generation IoT device <NUM> may determine whether the first intended activity is completed based on the data related to the prediction of completion of the first intended activity. The reminder generation IoT device <NUM> may generate the activity reminder to the user <NUM>, indicating to perform the first intended activity. The reminder generation IoT device <NUM> may generate the activity reminder to the user <NUM> on the nearby object to the user <NUM>. The nearby object may be identified based on at least one of data from the one or more UWB sensors <NUM> and the object map. The reminder generation IoT device <NUM> may identify the nearby object further based on one or more capabilities of the nearby object. Referring to the example <NUM> in <FIG>, the reminder generation IoT device <NUM> may determine that the user <NUM> has not moved towards the gas stove <NUM> for the pre-determined time period. The reminder generation IoT device <NUM> may determine the nearby object as the smart speaker <NUM>. The reminder generation IoT device <NUM> may determine that the smart speaker <NUM> has the audio capability. The smart speaker <NUM> may transmit the audio of the activity reminder as "Gas stove is on. Did you miss to operate the same?".

<FIG> illustrates a block diagram of a computer system according to an embodiment of the disclosure.

Referring to <FIG>, in an embodiment of the disclosure, the computer system <NUM> may be the reminder generation IoT device <NUM>. Thus, the computer system <NUM> may be used to generate the activity reminder in the loT environment <NUM>. The computer system <NUM> may comprise a central processing unit <NUM> (also referred as "CPU" or "processor"). The processor <NUM> may comprise at least one data processor. The processor <NUM> may include specialized processing units, such as integrated system (bus) controllers, memory management control units, floating point units, graphics processing units, digital signal processing units, or the like.

The processor <NUM> may be disposed in communication with one or more input/output (I/O) devices (not shown) via an I/O interface <NUM>. The I/O interface <NUM> may employ communication protocols/methods, such as, without limitation, audio, analog, digital, monoaural, RCA, stereo, institute of electrical and electronics engineers (IEEE) -<NUM>, serial bus, universal serial bus (USB), infrared, PS/<NUM>, BNC, coaxial, component, composite, digital visual interface (DVI), high-definition multimedia interface (HDMI), radio frequency (RF) antennas, S-Video, video graphics array (VGA), IEEE <NUM>. n /b/g/n/x, Bluetooth, cellular (e.g., code-division multiple access (CDMA), high-speed packet access (HSPA+), global system for mobile communications (GSM), long-term evolution (LTE), worldwide interoperability for microwave access (WiMax), or the like), or the like.

Using the I/O interface <NUM>, the computer system <NUM> may communicate with one or more I/O devices. For example, an input device <NUM> may be an antenna, keyboard, mouse, joystick, (infrared) remote control, camera, card reader, fax machine, dongle, biometric reader, microphone, touch screen, touchpad, trackball, stylus, scanner, storage device, transceiver, video device/source, or the like. An output device <NUM> may be a printer, fax machine, video display (e.g., cathode ray tube (CRT), liquid crystal display (LCD), light-emitting diode (LED), plasma, plasma display panel (PDP), organic light-emitting diode display (OLED) or the like), audio speaker, or the like.

The computer system <NUM> may communicate with the one or more receivers <NUM> through a communication network <NUM>. The processor <NUM> may be disposed in communication with the communication network <NUM> via a network interface <NUM>. The network interface <NUM> may communicate with the communication network <NUM>. The network interface <NUM> may employ connection protocols including, without limitation, direct connect, Ethernet (e.g., twisted pair <NUM>/<NUM>/<NUM> Base T), transmission control protocol/internet protocol (TCP/IP), token ring, IEEE <NUM>. 11a/b/g/n/x, or the like. The communication network <NUM> may include, without limitation, a direct interconnection, local area network (LAN), wide area network (WAN), wireless network (e.g., using wireless application protocol), the Internet, or the like. The network interface <NUM> may employ connection protocols include, but not limited to, direct connect, Ethernet (e.g., twisted pair <NUM>/<NUM>/<NUM> Base T), transmission control protocol/internet protocol (TCP/IP), token ring, IEEE <NUM>. 11a/b/g/n/x, or the like.

The communication network <NUM> includes, but is not limited to, a direct interconnection, an e-commerce network, a peer to peer (P2P) network, local area network (LAN), wide area network (WAN), wireless network (e.g., using wireless application protocol), the Internet, Wi-Fi, and such. The first network and the second network may either be a dedicated network or a shared network, which represents an association of the different types of networks that use a variety of protocols, for example, hypertext transfer protocol (HTTP), transmission control protocol/Internet protocol (TCP/IP), wireless application protocol (WAP), or the like, to communicate with each other. Further, the first network and the second network may include a variety of network devices, including routers, bridges, servers, computing devices, storage devices, or the like.

In some embodiments of the disclosure, the processor <NUM> may be disposed in communication with a memory <NUM> (e.g., a random access memory (RAM), a read-only memory (ROM), or the like not shown in <FIG>) via a storage interface <NUM>. The storage interface <NUM> may connect to the memory <NUM> including, without limitation, memory drives, removable disc drives, or the like, employing connection protocols, such as serial advanced technology attachment (SATA), integrated drive electronics (IDE), IEEE-<NUM>, a universal serial bus (USB), fiber channel, small computer systems interface (SCSI), or the like. The memory drives may further include a drum, magnetic disc drive, magneto-optical drive, optical drive, redundant array of independent discs (RAID), solid-state memory devices, solid-state drives, or the like.

The memory <NUM> may store a collection of program or database components, including, without limitation, a user interface <NUM>, an operating system <NUM>, a web browser <NUM>, or the like. In some embodiments of the disclosure, computer system <NUM> may store user/application data, such as, the data, variables, records, or the like, as described in this disclosure. Such databases may be implemented as fault-tolerant, relational, scalable, secure databases, such as Oracle® or Sybase®.

The operating system <NUM> may facilitate resource management and operation of the computer system <NUM>. Examples of operating systems include, without limitation, APPLE MACINTOSH® OS X, UNIXR, UNIX-like system distributions (E. , BERKELEY SOFTWARE DISTRIBUTION™ (BSD), FREEBSD™, NETBSD™, OPENBSD™, or the like), LINUX DISTRIBUTIONS™ (E. , RED HAT™, UBUNTU™, KUBUNTU™, or the like), IBM™ OS/<NUM>, MICROSOFT™ WINDOWS™ (XP™, VISTA™/<NUM>/<NUM>, <NUM>, or the like), APPLER IOS™, GOOGLER ANDROID™, BLACKBERRYR OS, or the like.

In some embodiments of the disclosure, the computer system <NUM> may implement the web browser <NUM> stored program component. The web browser <NUM> may be a hypertext viewing application, for example MICROSOFTR INTERNET EXPLORER™, GOOGLER CHROMETM0, MOZILLAR FIREFOX™, APPLER SAFARI™, or the like. Secure web browsing may be provided using Secure hypertext transport protocol (HTTPS), secure sockets layer (SSL), transport layer security (TLS), or the like. Web browsers <NUM> may utilize facilities, such as AJAX™, DHTML™, ADOBER FLASH™, JAVASCRIPT™, JAVA™, application programming interfaces (APIs), or the like. In some embodiments of the disclosure, the computer system <NUM> may implement a mail server (not shown in Figure) stored program component. The mail server may be an Internet mail server, such as Microsoft exchange, or the like. The mail server may utilize facilities, such as ASP™, ACTIVEX™, ANSI™ C++/C#, MICROSOFTR, NET™, CGI SCRIPTS™, JAVA™, JAVASCRIPT™, PERL™, PHP™, PYTHON™, WEBOBJECTS™, or the like. The mail server may utilize communication protocols, such as Internet message access protocol (IMAP), a messaging application programming interface (MAPI), MICROSOFTR exchange, post office protocol (POP), simple mail transfer protocol (SMTP), or the like. In some embodiments of the disclosure, the computer system <NUM> may implement a mail client stored program component. The mail client (not shown in Figure) may be a mail viewing application, such as APPLER MAIL™, MICROSOFTR ENTOURAGE™, MICROSOFTR OUTLOOK™, MOZILLAR THUNDERBIRD™, or the like.

Furthermore, one or more computer-readable storage media may be utilized in implementing embodiments consistent with the disclosure. Examples include a random access memory (RAM), a read-only memory (ROM), a volatile memory, a non-volatile memory, hard drives, a compact disc read-only memory (CD ROMs), digital video disc (DVDs), flash drives, disks, and any other known physical storage media.

Embodiments of the disclosure provides methods to select the messaging service by considering limitations of the session-based messaging service and the non-session-based messaging service. Further, various parameters related to the sender, the one or more receivers, the trigger message, and the communication network are considered for selecting the messaging service. Hence, appropriate messaging service is selected for communication the one or more messages between the sender and the one or more receivers.

Further, the limitations of the session-based messaging service, such as the network traffic, latency, and overhead are avoided by selecting the non-session-based messaging service based on the parameters. Hence, network is efficiently utilized.

Further, sessions are established only when there is requirement. Hence, network resources are effectively utilized. The disclosure establishes session prior to the initiation of a message from the sender, based on usage pattern. Hence, the latency in establishing the session is reduced. Further, user experience is also improved.

The disclosure provides methods that reminds the user to perform the intended activity in the loT environment. Further, these reminders are dynamic in nature. The disclosure avoids the need of pre-logging the activities or manually setting the reminders. In addition, the disclosure considers loT activities and non-loT activities. Further, the disclosure considers interrupts caused by loT events and non-loT events.

The terms "an embodiment", "embodiment", "embodiments", "the embodiment", "the embodiments", "one or more embodiments", "some embodiments", and "one embodiment" mean "one or more (but not all) embodiments of the disclosure(s)" unless expressly specified otherwise.

The terms "including", "comprising", "having" and variations thereof mean "including but not limited to", unless expressly specified otherwise.

The enumerated listing of items does not imply that any or all of the items are mutually exclusive, unless expressly specified otherwise. The terms "a", "an" and "the" mean "one or more", unless expressly specified otherwise.

A description of an embodiment with several components in communication with each other does not imply that all such components are required. On the contrary a variety of optional components are described to illustrate the wide variety of possible embodiments of the disclosure.

When a single device or article is described herein, it will be readily apparent that more than one device/article (whether or not they cooperate) may be used in place of a single device/article. Similarly, where more than one device or article is described herein (whether or not they cooperate), it will be readily apparent that a single device/article may be used in place of the more than one device or article, or a different number of devices/articles may be used instead of the shown number of devices or programs. The functionality and/or the features of a device may be alternatively embodied by one or more other devices which are not explicitly described as having such functionality/features. Thus, other embodiments of the disclosure need not include the device itself.

The illustrated operations of <FIG> shows certain events occurring in a certain order. In alternative embodiments of the disclosure, certain operations may be performed in a different order, modified, or removed. Moreover, steps may be added to the above described logic and still conform to the described embodiments. Further, operations described herein may occur sequentially or certain operations may be processed in parallel. Yet further, operations may be performed by a single processing unit or by distributed processing units.

Finally, the language used in the specification has been principally selected for readability and instructional purposes, and it may not have been selected to delineate or circumscribe the inventive subject matter. It is therefore intended that the scope of the disclosure be limited not by this detailed description, but rather by any claims that issue on an application based here on. Accordingly, the disclosure of the embodiments of the disclosure is intended to be illustrative, but not limiting, of the scope of the disclosure, which is set forth in the following claims.

Claim 1:
A method for generating an activity reminder by an electronic device (<NUM>) in an Internet of things, IoT, environment (<NUM>), the method comprising:
identifying (<NUM>) a movement of a user (<NUM>) towards an object (<NUM>, <NUM>, <NUM>);
predicting (<NUM>) a first intended activity of the user, associated with the object, based on past interactions of the user with the object, and a current state of the object , wherein the current state of the objected comprises a switched-on state;
determining (<NUM>) an occurrence of one or more interrupts for the first intended activity based on one or more parameters associated with a plurality of objects (<NUM>, <NUM>) in the loT environment, before performing the first intended activity associated with the object, wherein the one or more parameters comprise at least one of an audio parameter or a video parameter;
detecting (<NUM>) the movement of the user away from the object subsequent to the occurrence of the one or more interrupts and before performing the first intended activity; and
generating (<NUM>) an activity reminder to the user, indicating to perform the first intended activity when the user does not move towards the object for a pre-determined time period.