Patent Publication Number: US-2020293006-A1

Title: Home automation system for predicting a health change based upon a data trend and biometric characteristic and related methods

Description:
TECHNICAL FIELD 
     The present embodiments are directed to the field of electronics, and more particularly to home automation systems and related methods. 
     BACKGROUND 
     There are a number of home automation systems and approaches that seek to permit automated control of electrical devices in a house. The popularity of home automation has been increasing due to the greater availability of smartphones and tablets. As noted in “The Problem With Home Automation&#39;s Internet Of Things (IoT)”, and article appearing in Forbes dated Sep. 26, 2013, home automation was typically for wealthy consumers with an expensive system to control lights, home theater, security, air conditioning, and home audio. This market has expanded with many do it yourself (DIY) products now available, and, although the products are useful, they may be difficult to aggregate. In other words, as explained in the article, difficulties could arise if a consumer bought a Nest thermostat, Kwikset door lock, Phillips Hue lighting device, Lutron light switch, Sonos audio system, and Belkin wireless plugs. The consumer would need to have multiple applications each requiring time to setup, learn, and use. Additionally, the article states that there is no easy way to make devices work together, such as if the consumer wanted to trigger one event using one device based on another event from another device. 
     Multiple communication protocols may also be problematic. In particular, different devices may operate using different communication protocols, for example, Wifi, Zigbee, Zwave, Insteon, Itron, RadioRA2, and others. This may create additional difficulties for home automation. 
     One approach to address these shortcomings is for the consumer, which may include a user and/or enterprise, to use a service and device aggregator that provides one application and a consolidated wireless adapter unit. The user would contract with such a provider for multiple years. Unfortunately, as noted in the article, the consumer may not benefit from the most advanced hardware and software. 
     Another approach, as noted in the Forbes article, is to provide a single application that attempts to consolidate disparate applications and consolidate wireless adaptors, for example, using each of the different communications protocols. Still further improvements to the operation and integration of devices may be desirable. 
     SUMMARY 
     A home automation (HA) system may include at least one HA operation device and a biometric sensor. The HA system may also include an HA hub device to provide communications for the at least one operation device, and at least one controller. The at least one controller is configured to cooperate with the biometric sensor to monitor a biometric characteristic of a user, store historical operational data for the at least one HA operation device based upon the user, and determine a data trend of the at least one HA operation device based upon the stored historical operational data. The at least one controller may also be configured to correlate the data trend with the biometric characteristic of the user, and use machine learning to predict a health change of the user based upon the correlated data trend and biometric characteristic of the user. 
     The controller may be configured to correlate the data trend with diet data associated with the user, and use machine learning to predict the health change of the user also based upon the diet data. The diet data may include nutritional characteristics of food consumed by the user, for example. 
     The controller may be configured to correlate the data trend with medication data associated with the user, and use machine learning to predict the health change of the user also based upon the medication data, for example. The at least one controller may be configured to store the historical operational data for the at least one HA operation device based upon at least one other user. 
     The biometric characteristic may include at least one of a weight of the user, blood pressure of the user, heart rate of the user, and blood-oxygen level of the user. The at least one HA operation device may include a pedometer, for example. 
     The at least one controller may be carried by the HA hub device. The at least one controller may include a cloud server remote from the HA hub device in a cloud computing environment, for example. 
     The HA system may also include at least one HA user interface device configured to wirelessly communicate with the at least one HA operation device. The at least one HA operation device may include at least one Internet of Things (IoT) device, for example. 
     A method aspect is directed to a method of predicting a health change of a user of a home automation (HA) system that includes at least one HA operation device, a biometric sensor, an HA hub device to provide communications for the at least one HA operation device. The method may include using at least one controller to cooperate with the biometric sensor to monitor a biometric characteristic of the user, and store historical operational data for the at least one HA operation device based upon the user. The method may also include using the at least one controller to determine a data trend of the at least one HA operation device based upon the stored historical operational data, and correlate the data trend with the biometric characteristic of the user. The method may further include using the at least one controller to use machine learning to predict the health change of the user based upon the correlated data trend and biometric characteristic of the user. 
     A computer readable medium aspect is directed to a non-transitory computer readable medium for predicting a health change of a user of a home automation (HA) system that includes at least one HA operation device, a biometric sensor, an HA hub device to provide communications for the at least one HA operation device. The non-transitory computer readable medium includes computer executable instructions that when executed by at least one controller cause the at least one controller to perform operations. The operations may include cooperating with the biometric sensor to monitor a biometric characteristic of the user, and storing historical operational data for the at least one HA operation device based upon the user. The operations may also include determining a data trend of the at least one HA operation device based upon the stored historical operational data, and correlating the data trend with the biometric characteristic of the user. The operations may further include using machine learning to predict the health change of the user based upon the correlated data trend and biometric characteristic of the user. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1 a    is a schematic diagram of an electronic device integration system in accordance with an embodiment of the present invention. 
         FIG. 1 b    is a schematic diagram of an HA system in accordance with an embodiment. 
         FIG. 2 a    is a schematic block diagram of a message queue for use in the system of  FIG. 1   a.    
         FIG. 2 b    is a schematic block diagram of an HA system including message queues in accordance with an embodiment. 
         FIG. 3  is a schematic diagram of an action server for use in the system of  FIG. 1   a.    
         FIG. 4  is a schematic diagram of operation of an analytics server for use in the system of  FIG. 1   a.    
         FIG. 5  is a schematic diagram of a camera server for use in the system of  FIG. 1   a.    
         FIG. 6  is a schematic diagram of a configuration server for use in the system of  FIG. 1   a.    
         FIG. 7  is a schematic diagram of a debug server for use in the system of  FIG. 1   a.    
         FIG. 8 a    is a schematic diagram of a discovery server for use in the system of  FIG. 1   a.    
         FIG. 8 b    is another schematic diagram of the discovery server of  FIG. 8   a.    
         FIG. 9  is a schematic diagram of a notification server for use in the system of  FIG. 1   a.    
         FIG. 10  is a schematic diagram of a loader server for use in the system of  FIG. 1   a.    
         FIG. 11  is a schematic diagram of a status server for use in the system of  FIG. 1   a.    
         FIG. 12  is a schematic diagram of a web server for use in the system of  FIG. 1   a.    
         FIG. 13 a    is a schematic diagram of a security server in the system of  FIG. 1   a.    
         FIG. 13 b    is another schematic diagram of a security server in accordance with an embodiment. 
         FIG. 14 a    is a diagram of a user interface displaying contextual help on a remote device of the system of  FIG. 1   a.    
         FIG. 14 b    is a diagram of a user interface displaying contextual help on a remote device of the system of  FIG. 1   a.    
         FIG. 15 a    is a diagram of a user interface showing addressable devices arranged by room on a remote device of the system of  FIG. 1   a.    
         FIG. 15 b    is a diagram of a user interface showing addressable devices arranged by device type on a remote device of the system of  FIG. 1   a.    
         FIG. 15 c    is a diagram of a user interface showing addressable devices arranged by scene type on a remote device of the system of  FIG. 1   a.    
         FIG. 16  is a diagram of a user interface showing a color picker for use with an LED light addressable device of the system of  FIG. 1   a.    
         FIG. 17  is a schematic block diagram of a remote device and an LED light bulb addressable device in accordance with an embodiment of the present invention. 
         FIG. 18  is a schematic diagram of an interface between multiple hub devices in accordance with an embodiment of the present invention. 
         FIG. 19  is a schematic diagram of bridges in the system of  FIG. 1   a.    
         FIG. 20  is a schematic diagram of operation of system of  FIG. 1 a    when a new bridge is added. 
         FIG. 21 a    is a diagram illustrating sandboxed processes in the system of  FIG. 1   a.    
         FIG. 21 b    is another schematic diagram illustrating sandboxed processes in the system of  FIG. 1   a.    
         FIG. 22  is a diagram illustrating a responsive scene definition in the system of  FIG. 1   a.    
         FIG. 23  is a flow diagram illustrating ingredient responsive scenes in the system of  FIG. 1   a.    
         FIG. 24  is a diagram of a user interface showing recommended purchases based upon ingredients to complete a scene in the system of  FIG. 1   a.    
         FIG. 25  is a diagram of a user interface showing the ability of a user to choose from a list of ingredient blocks for a scene in the system of  FIG. 1   a.    
         FIG. 26  is a diagram of a user interface showing suggested device operation blocks based upon user input for a scene in the system of  FIG. 1   a.    
         FIG. 27  a diagram of a user interface showing a prompt for user input to choose what device provides an ingredient for a scene in the system of  FIG. 1   a.    
         FIG. 28  is a diagram of a user interface showing different scenes for a given set of ingredients or devices in the system of  FIG. 1   a.    
         FIG. 29 a    is a diagram of a user interface showing a prompt for user input to choose devices to map a scene to devices specific to a home in the system of  FIG. 1   a.    
         FIG. 29 b    is a schematic block diagram of operation of an HA device scene controller in the HA system of  FIG. 1   a.    
         FIG. 30 a    is a block diagram of an electronic device integration system in accordance with another embodiment of the present invention. 
         FIG. 30 b    is a schematic diagram of an HA system for generating a user health score in accordance with an embodiment. 
         FIG. 31  is a diagram of a hub device for detecting proximity to a remote device in accordance with an embodiment of the present invention. 
         FIG. 32  is a schematic diagram of multiple electronic device integration systems in accordance with an embodiment of the present invention. 
         FIG. 33  is a schematic diagram of an electronic device integration system including a short-range communication protocol ID device in accordance with an embodiment of the present invention. 
         FIG. 34  is a diagram of a user interface illustrating event generation on a remote device for multiple electronic device integration systems in accordance with an embodiment of the present invention. 
         FIG. 35  is a schematic diagram of a climate control system in accordance with an embodiment. 
         FIG. 36  is a graph illustrating operation of the climate control system of  FIG. 35 . 
         FIGS. 37 a -37 e    are schematic block diagrams of an HA system illustrating communications between an addressable HA device and a remote access wireless communications device in accordance with an embodiment. 
         FIG. 38  is a schematic diagram of an HA system in accordance with an embodiment. 
         FIG. 39  is a schematic block diagram of the HA system of  FIG. 38 . 
         FIG. 40  is a flow chart illustrating operation of the controller of the HA system of  FIG. 39 . 
         FIG. 41  is a schematic block diagram of an HA system in accordance with another embodiment. 
         FIG. 42  is a schematic block diagram of an HA system in accordance with another embodiment. 
         FIG. 43  is a schematic block diagram of an HA system in accordance with another embodiment. 
         FIG. 44  is a schematic block diagram of an HA system in accordance with another embodiment. 
         FIG. 45  is a schematic block diagram of an HA system in accordance with another embodiment. 
         FIG. 46  is a schematic block diagram of another embodiment of an HA system. 
         FIG. 47  is a flow chart illustrating operation of the controller of the HA system of  FIG. 46 . 
         FIG. 48  is a schematic block diagram of an HA system in accordance with another embodiment. 
         FIG. 49  is a schematic block diagram of an HA system in accordance with another embodiment. 
         FIG. 50  is a schematic block diagram of an HA system in accordance with another embodiment. 
         FIG. 51  is a schematic block diagram of an HA system in accordance with another embodiment. 
         FIG. 52  is a schematic block diagram of another embodiment of an HA system. 
         FIG. 53  is a flow chart illustrating operation of the controller of the HA system of  FIG. 46 . 
         FIG. 54  is a schematic block diagram of an HA system in accordance with another embodiment. 
         FIG. 55  is a schematic block diagram of an HA system in accordance with another embodiment. 
     
    
    
     DETAILED DESCRIPTION 
     The present invention will now be described more fully hereinafter with reference to the accompanying drawings, in which preferred embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Like numbers refer to like elements throughout, and prime notation is used to indicate similar elements in alternative embodiments. Prime and multiple prime notations are used to refer to like elements in different embodiments. 
     Referring initially to  FIG. 1 a   , an electronic device integration system is illustratively in the form of a home automation (HA) system  20 , and which is referred to as the K4Connect system. The HA system  20  illustratively includes a plurality of addressable devices  31   a - 31   n , a home device  32 , a remote device  36 , and cloud device  33 . While an HA system  20  is described herein, it should be understood that the system is not limited to use in a home and may be used in any setting, commercial, industrial, residential, etc. 
     Addressable devices  31   a - 31   n  may include controllable devices and/or sensors, for example, a motion detector, thermostat, light switch, audio controller, door lock, and/or camera. Of course, the addressable devices may include additional or other devices. 
     While a cloud device  33  or hardware server is described, it should be understood by those skilled in the art that the processes and functions performed by the could device may be performed by a processor  46  or by multiple processors in different geographic locations and over different networks in what is understood by those skilled in the art as the cloud. The home device  32  may be a personal computer, tablet computer, standalone computing device, or any other computing device. The HA system  20  may also include a hub device  34  (i.e., a K4Hub). In some embodiments, the hub device  34  and the home device  32  may be within a home  47  and wirelessly connected to a home network, which may provide communication with the Internet. The functions and interconnections of these devices within the system will be described in further detail below. 
     The K4Home software program runs the K4Connect HA system  20  of home, office, business, and building automation for addressable devices  31   a - 31   n  that can be connected into the program. The K4Home software is available as a software only package that can be loaded onto a personal computer or other small computer devices, for example, the home device  32 . The functions of the K4Home software are executed by respective processors or processing circuitry on one or more devices running the K4Home software, for example, a processor  38  of the home device  32  as will be described below. 
     The K4Hub  34  is a device that may also run the K4Home software and hosts the system architecture on the device. The K4Hub  34  includes a housing  41  and hub processing circuitry  42  carried by the housing. The K4Hub  34  also includes a plurality of radio ports  43   a - 43   n , for example, universal serial bus (USB) ports carried by the housing  41  and for coupling to any of a plurality of radio controllers  44   a - 44   n . The K4Hub  34  runs the system locally and can communicate with the addressable devices  31   a - 31   n  directly instead of routing through a cloud based process. In other words, the hub processing circuitry  42  cooperates with radio controllers  44   a - 44   n  that are plugged in to communicate with addressable devices  31   a - 31   n  based upon the respective protocols. 
     The radio controllers  44   a - 44   n  may each be for a given radio protocol. For example, a Z-wave radio controller may be plugged into one of the radio ports  43   a - 43   n , which allows the K4Hub  34  to communicate with Z-wave based addressable devices. A second or third radio controller may be plugged into the radio ports  43   a - 43   n  of the K4Hub for adding the ability to communicate with controllable devices using second and third radio protocols. 
     The K4Hub  34  is an improvement on current technology since it reduces the latency and system failures common on current home automation devices that require a network connection. Similarly to the K4Hub  34 , the K4Home software running on a personal computer, for example, the home device  32 , can be augmented with additional home automation communication protocols such as ZigBee and Z-Wave by attaching ports through the K4Hub or computer&#39;s USB port. 
     The K4App is the location of the user interface  35  of the K4Connect HA system  20  and allows the user to access the K4Home software and control the K4Connect system  20  through or from the remote device  36 , for example, a smartphone or tablet device that includes a display  48  and a processor  49  coupled to the display. The user interface  35  may be also accessed by a desktop application for a personal computer and/or by an on-screen application for a television. There may be more than one remote device  36  and each remote device may be a different type of device. 
     In some embodiments, the remote device  36  may connect “locally” without communicating through or with the cloud device  33 . This may be particularly advantageous because communication may not rely on network connectivity and function locally independent of the Internet. Additionally, communication may be relatively faster and more reliable. 
     The remote server or cloud device  33 , which runs software referred to as K4Away, is a cloud-based subscription system that provides the connection between the local K4Home software, for example, running on the home device  32  or K4Hub  34 , and the K4App when outside of the local home network, for example running on the remote device  36 . K4Away also provides the connection between the K4Home software and K4Connect system analytics and help system. The K4App may, in some embodiments, connect directly to K4Home, i.e., the home device  32  or K4Hub  34 , without communicating through the cloud device  33  or indirectly without communication through the cloud device. 
     Referring now to  FIG. 1 b   , the above-described components of the HA system  20  will be described. The HA system  20  includes addressable HA devices  31   a - 31   n , each configured to wirelessly communicate using a respective HA wireless communications protocol from among different HA wireless communications protocols. The addressable HA devices  31   a - 31   n  may include any of motion detectors, thermostats, light switches, audio controllers, door locks, and/or cameras. Of course, the addressable HA devices  31   a - 31   n  may include other and/or additional devices. 
     The HA system  20  also includes HA wireless radio controllers  44   a - 44   n , each configured to wirelessly communicate using a respective different HA wireless communications protocol also from among the different HA wireless communications protocols. Each HA wireless radio controller  44   a - 44   n  includes circuitry  441   a - 441   n  and a connector  442   a - 442   n  coupled thereto. The HA wireless radio controllers  44   a - 44   n  may be Zigbee controllers, Z-Wave controllers, and/or other types of controllers, for example. 
     The HA system  20  also includes an HA hub device  34  that includes a housing  41  and wireless radio port connectors  43   a - 43   n  carried by the housing. Each port connector  43   a - 43   n  is configured to couple to a respective connector  442   a - 442   n  of a corresponding HA wireless radio controller  44   a - 44   n . The port connectors  43   a - 43   n  may be USB connectors, for example, and/or other or additional types of connectors. The HA hub device  34  also includes hub processing circuitry  42  coupled to the wireless radio port connectors  43   a - 43   n . The hub processing circuitry  42  communicates with the addressable HA devices  31   a - 31   n  based upon the respective HA wireless communications protocols. In some embodiments, the HA wireless radio controllers  44   a - 44   n  may communicate directly with the addressable devices via the HA hub device  34 , for example, instead of routing through a cloud based process, as will be appreciated by those skilled in the art. 
     A method aspect is directed to a method of communicating in the HA system  20 . The method includes using HA wireless radio controllers  44   a - 44   n  to wirelessly communicate using a respective different HA wireless communications protocol from among the different HA wireless communications protocols. The method also includes using the HA hub device  34  to communicate with the addressable HA devices  31   a - 31   n  based upon the respective HA wireless communications protocols. 
     Referring now additionally to  FIG. 2 a   , the primary functions of the HA system  20  (i.e., K4Connect) are based around an independent standalone message queue server  50  that is a combination of an independent local message queue  51  located on a device running the K4Home software and a cloud message queue  52  hosted on the cloud device  33  (i.e. K4Away), which provides connectivity to registered devices outside the local home network. Communication between the message queues  51 ,  52  and connected addressable devices  31   a - 31   n , connected servers, and connected bridges use web sockets as the transport medium, for example. 
     Both the local and cloud message queues  51 ,  52  function independently but remain continuously connected so that no matter the user location, communication to and from the connected device, e.g., servers, and bridges is still available. The continuous connection is initiated from the local message queue  51  to reduce security issues that may be inherent when piercing a firewall of a local network. Having the connection originate from inside the firewalled system, for example, allows for the message queues  51 ,  52  to more easily connect while maintaining the security integrity of the home system. In other words, each remote device  36  connects to the cloud message queue  52  and not directly to the local message queue  51  or any of the K4Home  32  or K4Hub  34 . Additionally, communication between the local message queue  51  and the cloud message queue  52 , the connected addressable devices  31   a - 31   n , servers, and bridges may be SSL encrypted including on the local network for increased security. When the K4App, for example, via the remote device  36 , is connected to the cloud or remote server  33 , the continuous connection allows for the user&#39;s connection to the cloud server to serve as a direct connection to the local message queue  51 . 
     The local message queue  51  receives and distributes messages to and from the cloud message queue  52  and to and from the local servers  81  and device bridges  82 . This distribution technique for the messages allows for independence for each component of the program and leaves the logic or prescribed action to the individual servers or bridges. This independence of the components of the program may also reduce the probability of system crashing errors. This also allows for continuous rolling out of new bridges and compatibility of new devices without updating the complete software package, for example. 
     As will be appreciated by those skilled in the art, a typical prior art automation integration system exchanges messages either all within the home network or all on the Internet by penetrating through a firewall. The embodiments described herein advantageously provide a hybrid messaging approach that includes the increased speed of “in-home” message processing (processing via the Internet adds delay) and has the increased security of the Internet (does not penetrate a firewall to expose the home network). 
     Referring now to  FIG. 2 b   , another aspect of the HA system  20  with respect to the local and cloud message queues  51 ,  52  will now be described. The HA system  20  includes addressable HA devices  31   a - 31   n  each having associated therewith a respective device capability, device configuration, and device state. 
     Each device configuration may include at least one of a device address, a device location, and a device identifier, for example. Exemplary device configurations may include an IP address of the device, the location of the device within a house, and channel location (e.g., left, right) in an audio configuration. Of course, the device configuration may include other and/or additional elements. 
     Each device capability may include at least one of a sensing function, and an output function. For example, with respect to a light switch, the device capability may include the capability to be “on”, “off”, and be at different “dimmer levels.” 
     Each device state may include a current state from among a plurality of possible states. For example, with respect to a light switch, the current state may be “on”, “off”, and “dimmed to a given level.” 
     The HA system  20  includes a cloud message queue controller  521  and a cloud message queue memory  522  coupled thereto in the cloud for storing the device configurations, device capabilities, and device states for the plurality of addressable HA devices  31   a - 31   n . The cloud message queue controller  521  and the cloud message queue memory  522  may be part of the cloud message queue  52 , for example. 
     The HA system  20  also includes a home device message queue controller  511  and a home device message queue memory  512  coupled thereto for storing the device configurations, device capabilities, and device states for the plurality of addressable HA devices  31   a - 31   n . The home device message queue controller  511  and the home device message queue memory  512  may be part of the local message queue  51 , for example. 
     The cloud message queue controller  521  and the home device message queue controller  511  synchronize device configurations, device capabilities, and device states for the addressable HA devices  31   a - 31   n . The cloud message queue controller  521  exchanges messages with the local message queue controller  511  relating to the addressable HA devices, for example, for communication with the addressable devices  31   a - 31   n  and for synchronization. For example, such messages may include messages related to the operation and control of the addressable HA devices  31   a - 31   n.    
     A local client device  36   a  or remote device (e.g., running K4App) includes a local client device controller  361   a  and local client device memory  362   a  coupled thereto for storing the device configurations, device capabilities, and device states for the addressable HA devices  31   a - 31   n  upon synchronization with the local message queue controller  511 . The local client device controller  361   a  exchanges messages with the local message queue controller  511  relating to the addressable HA devices  31   a - 31   n , for example, sensing, response, and control operations. 
     A cloud client device  36   b  or remote device (e.g., running K4App) includes a cloud client device controller  361   b  and cloud client device memory  362   b  coupled thereto for storing the device configurations, device capabilities, and device states for the addressable HA devices  31   a - 31   n  upon synchronization with the cloud message queue controller  521 . The cloud client device controller  361   b  exchanges messages with the cloud message queue controller  521  relating to the addressable HA devices  31   a - 31   n , for example, sensing, response, and control operations. 
     As will be appreciated by those skilled in the art, by synchronizing the device configurations, device capabilities, and device states for the addressable HA devices  31   a - 31   n , or messages, communication with a cloud or local client device  36   a ,  36   b  may be quicker as processing of the messages, responses, status queries, instructions, etc., for example, can be processed at the cloud or local client device or at the nearest of the cloud or message queue (i.e., the request or communication generally may not have to travel to one or the other of the local or cloud message queues  51 ,  52 ). 
     A method aspect is directed to a method of communicating with a plurality of addressable HA devices  31   a - 31   n  each having associated therewith a respective device capability, device configuration, and device state. The method includes using a cloud message queue controller  521  and a cloud message queue memory  522  coupled thereto in the cloud for storing the device configurations, device capabilities, and device states for the plurality of addressable HA device. The method also includes using a home device message queue controller  511  and a home device message queue memory  512  coupled thereto for storing the device configurations, device capabilities, and device states for the plurality of addressable HA devices. The cloud message queue controller  521  and the home device message queue controller  511  synchronize device configurations, device capabilities, and device states for the plurality of addressable HA devices  31   a - 31   n.    
     Referring now additionally to  FIGS. 4-13 , the K4Home program, for example executed using the home device  32  or K4Hub  34 , provides for independent servers or functional modules for each of the functions of the HA system  20 . The servers  81  are separated from the bridges  82  running on the HA system  20  for security and may allow independent running of the system as a whole. The servers  81  on the home automation integration system  20  include an action server  69 , analytics server  54 , camera server  61 , configuration server  62 , debug server  63 , discovery server  55 , loader server  64 , message server  65 , notification server  66 , status server  67 , update server  59 , web server  68 , and security server  56 . More servers can be added to the software if new functions are needed. While the term server has been used herein, it should be understood that a server may be one or more standalone software processes that are executed on one or more processors on any device, for example, as described above. The functionality of each server  81  is performed by a processor, controller, and/or related circuitry, particularly on the device which it is executed, for example, the home device processor  38  or the hub device processing circuitry  42 , as will be appreciated by those skilled in the art. 
     The action server is continuously running on the HA system  20 , and more particularly, the home device  32 , and executes the responsive scenes of the K4Home system or components within the home ( FIG. 3 ). The analytics server  54  logs user and system actions to the cloud storage system or server  33  and receives suggestions of possible responsive scenes the user could implement or actions the user could take to improve their K4Home HA system  20  ( FIG. 4 ). 
     In the initial K4Home system setup, the analytics server  54  requests advertisements from the servers  81  and bridges  82  on the system. The servers  81  and bridges  82  on the K4Home system  20  return advertisements, which allows for the analytics server  54  to subscribe to the individual servers and bridges. Once subscribed, the servers and bridges  82  send individual events, commands, and variable changes to the analytics server  54 , which keeps a log of the data sent. 
     At intervals, which may be periodic or regular, the analytics server  54  reports the data collected to the cloud system or cloud device  33  via a private globally unique identifier (GUID). The cloud-based analytics or device  33  processes and reviews the anonymized data, storing the data in a cloud database. This data is then used to review the functions of the K4Home HA system  20  which may reveal any problems that may exist in the software. This HA system  20  can also use the data gathered from the security server to assess any security threats and develop mitigation plans. The cloud-based analysis or cloud device  33  also reviews the K4Home system and recommends devices and responsive scenes to the private GUIDs. Once the information in the cloud has been analyzed and gathered by the cloud device  33  it is pushed back to the local analytics server  54  with the next time to “check-in” to the cloud. 
     The camera server  61  ( FIG. 5 ) locates camera images/video and streams the images/video to the system. The camera server also acts as an image proxy for remote users not able to directly connect to the camera, for example. 
     The configuration server  62  ( FIG. 6 ) stores the persistent configuration of the home automation integration system  20 . The configuration server  62  also uses the device descriptions during the device connection process to setup addressable devices  31   a - 31   n  on the HA system  20  in tandem with a device setup wizard. The debug server  63  enables bridge debugging ( FIG. 7 ). 
     The discovery server  55  ( FIG. 8 a   ) finds addressable devices to connect to the K4Connect system  20 . The discovery server  55  uses signatures of devices, for example, addressable devices  31   a - 31   n  in its search to discover devices that are not natively discoverable for connection to the system. With respect to typical prior art home automation integration systems, certain addressable devices do not automatically broadcast their availability and thus have to be manually connected by the user. Manual entry often involves advanced technical knowledge or having to follow detailed complicated instructions to add the device to their home automation systems, for example, manually entering an IP address, device ID, and/or other identifying information. The discovery server  55  reduces these complications. 
     Example code executed on the discovery server  55  with respect to a network device and a USB device, respectively, are below: 
     
       
         
           
               
             
               
                   
               
             
            
               
                 &lt;signature cls=″com.k4connect.someNetworkDevice″ 
               
               
                 description=”Example Network Device″&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;mdns&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;services&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;service&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;name&gt;DeviceName.*&lt;/name&gt; 
               
               
                   
                 &lt;type&gt;http&lt;/type&gt; 
               
               
                   
                 &lt;protocol&gt;tcp&lt;/protocol&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;/service&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;/services&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;/mdns&gt; 
               
               
                   
                 &lt;upnp&gt; 
               
               
                   
                 &lt;deviceType&gt;urn:Manufacturer:device:sensor:1&lt;/deviceType&gt; 
               
               
                   
                 &lt;/upnp&gt; 
               
               
                   
                 &lt;macs&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;mac&gt;ff:ff:ff&lt;/mac&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;/macs&gt; 
               
            
           
           
               
            
               
                 &lt;/signature&gt; 
               
               
                 &lt;signature cls=″com.k4connect.someUsbDevice″ 
               
               
                 description=“Example USB Device″&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;udev&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;devices&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;device&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;attributes&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;attribute name=″ID_VENDOR_ID″ 
               
            
           
           
               
               
            
               
                   
                 pattern=″10c4″/&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;attribute name=″ID_MODEL_ID″ 
               
            
           
           
               
               
            
               
                   
                 pattern=″ea60″/&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;attribute name=″DEVNAME″ 
               
            
           
           
               
               
            
               
                   
                 pattern=″{circumflex over ( )}\/dev\/ttyUSB\d+″/&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;/attributes&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;/device&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;/devices&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;/udev&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;/signature&gt; 
               
               
                   
                   
               
            
           
         
       
     
     The discovery server  55  is typically always running processes that monitor the system home automation integration  20  either passively waiting for a signal from a new controllable device or scanning the system for signatures of the addressable devices  31   a - 31   n . The discovery server  55  runs UPNP and MDNS processes that use a text match process from the signatures of the addressable devices  31   a - 31   n  to identify the controllable device. The discovery server  55  also runs multicast processes and connects to these unconnected addressable devices  83 , for example, by a challenge response. 
     The advantageous elements of the discovery server  55  are the ARP scan and the udev scan. The ARP scan runs a port match for loaded controllable device signatures and runs a challenge-response process to identify the addressable devices  31   a - 31   n . For example, discovery server  55  may query a port with data and get an identifying response based upon the query. The ARP scan also identifies the device by MAC address matching. The other advantageous element is the UDEV scan which uses a USB match for devices connected to the hardware running K4Home and running a TTY Match, which identifies the device with a challenge response process. As will be appreciated by those skilled in the art, any number of elements or network characteristics that define a controllable device signature may be used. 
     Once the discovery server  55  has discovered a new addressable device (i.e., new to the system  20 ), it sends notifications over the message queue  51  to the configuration server  62  and notification server  66  ( FIG. 7 ), which then notify the user of the newly discovered addressable device and begins a wizard set-up process. When a new addressable device becomes available (e.g., new to market and not just the system) for which there is not an identifiable signature, a new signature filter may be added to the discovery server  55 . 
     In some embodiments, advertising from the addressable devices  31   a - 31   n  may be used to limit an addressable device. For example, a controllable speaker device may appear to the home automation integration system  20  as a generic device based upon advertising. However, a query based upon a subset of addresses or signature elements may be used, which may increase the speed of controllable device discovery. For example, signature elements may be used to limit or restrict a device type, and discovery may continue based upon the subset. 
     Referring now additionally to  FIG. 8 b   , the discovery server  55  will be described with respect to the HA system  20 . The addressable HA devices  31   a - 31   n  each have a respective HA device signature associated therewith and each is configured to wirelessly communicate using respective different wireless communications protocols from among different wireless communications protocols. The addressable HA devices  31   a - 31   n  may include any of motion detectors, thermostats, light switches, audio controllers, door locks, and/or cameras. Of course, the addressable HA devices  31   a - 31   n  may include other and/or additional devices. 
     The discovery server  55  may be in the form of a controller  551  and a memory  552  coupled thereto. The memory  552  stores HA device signatures for paired and unpaired ones of the addressable HA devices  31   a - 31   n . The HA device signatures may include, for example, MAC addresses, port data, and/or universal serial bus (USB) identifiers. 
     The controller  551  polls the addressable HA devices  31   a - 31   n  and determines an unpaired addressable HA device from among the plurality thereof based upon the polling. The controller  551  may poll the addressable HA devices  31   a - 31   n  by polling for a broadcast from the addressable HA devices and/or by scanning for addressable devices responsive to a given one of stored HA device signatures stored in the memory  552 . 
     The controller  551  also compares the associated HA device signature of the unpaired addressable HA device with the stored HA device signatures. The controller  551  may compare the associated HA device signature of the unpaired addressable HA device with the stored HA device signatures based upon at least one of a universal plug and play (UPnP) process and a multicast domain name system (mDNS) process. Any of the UPnP and mDNS processes may be executed based upon a text match process, for example. 
     In some embodiments, the addressable HA devices  31   a - 31   n  may each have port data associated therewith, in which case the controller  551  may poll the addressable HA devices based upon an address resolution protocol (ARP) scan, and compare the associated HA device signature of the unpaired addressable HA device with the stored HA device signatures based upon port data from the ARP scan. 
     Alternatively or additionally, the controller  551  may poll the addressable HA devices  31   a - 31   n  based upon a udev scan, in which case the controller compares the associated HA device signature of the unpaired addressable HA device with the stored HA device signatures based upon the udev scan. 
     The controller  551 , when there is a match between the HA device signature of the unpaired addressable HA device and one of the stored HA device signatures, permits pairing of the unpaired addressable HA device to communicate with the unpaired addressable HA device using the respective wireless communications protocol. The controller  551  may prompt a user to approve pairing of the unpaired addressable HA device. The pairing of the unpaired addressable HA device may be based upon a challenge response from an electronic device associated with a user, for example, the remote device  36 . 
     A communications interface  553  provides communication between the controller  551  and the cloud, for example, the cloud device  33 . The controller  551  communicates with the cloud device  33  via the communications interface  553  to update the stored HA device signatures in the memory  552 . 
     The HA system  20  also includes radio controllers  44   a - 44   n  coupled to the controller  551 . Each of the addressable devices  31   a - 31   n  is configured to wirelessly communicate with the controller  551  via respective radio controllers  44   a - 44   n.    
     A method aspect is directed to a method of permitting pairing of unpaired addressable HA devices  31   a - 31   n  in the HA system  20 . The method includes using the  551  controller and the memory  552  coupled thereto storing a plurality of HA device signatures for paired and unpaired ones of plurality of addressable HA devices to poll the plurality of addressable HA devices and determine an unpaired addressable HA device from among the plurality thereof based upon the polling. The controller  551  and the memory  552  are also used to compare the associated HA device signature of the unpaired addressable HA device  31   a - 31   n  with the stored HA device signatures, and when there is a match between the HA device signature of the unpaired addressable HA device and one of the stored HA device signatures, permit pairing of the unpaired addressable HA device to communicate with the unpaired addressable HA device using the respective wireless communications protocol. 
     The loader server  64  loads bridges  82  and servers  81  ( FIG. 10 ). The message server  65  runs or operates the message queue  51 . The notification server  66  sends notifications from the system  20  to the user interface  35 , for example, at the remote device  36  ( FIG. 9 ). 
     The status server  67  serves as a system wide state machine storing a log of last known state without having to poll devices in the system ( FIG. 11 ). This is accomplished by having the status server  67  perform as a standalone state machine tracking the last known states of the system. As will be appreciated by those skilled in the art, the status server  67  is advantageously an improvement relative to the current common practice where the system stores the state information in the driver stack, for example. The web server  68  runs the user interface content ( FIG. 12 ). In some embodiments, the user interface content may be stored locally. 
     The security server  56  executes security processes of the home automation integration system  20  ( FIG. 13 a   ). The security server  56  listens on open communication ports  84  not being used by the home automation integration system  20 . This allows the security server  56  to log when a device, for example, an addressable device  31   a - 31   n  or remote device  36 , scans or connects to the port. The security server  56  may then ignore any device that is known to scan or connect and is not a threat to the system and log when it receives an unknown or unexpected scan. For example, an open port may be scanned by or connected to a connected user&#39;s remote iPhone, but since this is an expected action from an iPhone, the security server  56  does not automatically consider this a threat to the home automation integration system  20 . If a connected home automation or addressable device  31   a - 31   n , for example, a refrigerator, does the same scan of or connects to the open ports, the security server  56  logs the action, and then reports the logs to the analytics server  54 . The security server  56  is aware of what can be considered normal behavior for an addressable device  31   a - 31   n  by way of a signature file included for all known controllable devices. In other words, because the types of devices, both remote and controllable, coupled to the home automation integration system  20  on network are known, traffic among the devices can be monitored to maintain security. If traffic or communications associated with a particular device is determined to be erratic, the security server  56  may identify the device as being hijacked and/or malware and flagged for reporting to the analytics server  54 . The analytics server  54  uploads the data to the cloud device  33  for security analysis. 
     An example security server signature that describes what may be considered normal behavior of a network device is below: 
     
       
         
           
               
               
             
               
                   
                   
               
             
            
               
                   
                 &lt;signature cls=″com.k4connect.someNetworkDevice″ 
               
               
                   
                 description=”Example Network Device″&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;behavior&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;http&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;url&gt;http://api.someurl.com/*&lt;/url&gt; 
               
               
                   
                 &lt;frequency&gt;300&lt;/frequency&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;/http&gt; 
               
               
                   
                 &lt;socket&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;destination&gt;*&lt;/destination&gt; 
               
               
                   
                 &lt;port&gt;80&lt;/port&gt; 
               
               
                   
                 &lt;quantity&gt;3&lt;/quantity&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;/socket 
               
            
           
           
               
               
            
               
                   
                 &lt;/behavior&gt; 
               
            
           
           
               
               
            
               
                   
                 &lt;/signature&gt; 
               
               
                   
                   
               
            
           
         
       
     
     The cloud server or cloud device  33  may perform an analysis to assess or classify patterns and recommend actions for the security server  56 . Some examples of actions for the security server  56  to take include notifying the user of abnormal actions of a device, disconnecting a compromised device from the K4Connect system  20 , or ignoring if the action is not malicious. The K4Connect system  20  in some instances may recognize a vulnerability or attack in a manufacturer&#39;s smart device and can provide the information about the vulnerability to the manufacturer. Of course, the cloud device  33  may recommend other and/or additional actions for the security server based upon the analysis. 
     Referring now to  FIG. 13 b   , the security server  56  with respect to the HA system  20  will now be described. The HA system  20  includes addressable HA devices  31   a - 31   n  each having a respective HA device signature associated therewith, which may be stored in a memory  562 . The HA device signatures may include data regarding expected actions of the addressable HA devices  31   a - 31   n . The HA device signatures may also include MAC addresses, port data, and universal serial bus (USB) identifiers, for example. Of course, the HA device signatures may include any combination of and/or or additional identifiers that may be used as a basis to characterize operating behavior of the addressable HA devices  31   a - 31   n.    
     The addressable HA devices  31   a - 31   n  may include any of motion detectors, thermostats, light switches, audio controllers, door locks, and/or cameras. Of course, the addressable HA devices  31   a - 31   n  may include other and/or additional devices. The addressable devices  31   a - 31   n  wirelessly communicate using respective different wireless communications protocols from among different wireless communications protocols. 
     The HA system  20  includes an HA security controller  561  coupled to the memory  562  and that communicates with the addressable HA devices  31   a - 31   n  via respective communications ports, for example by scanning or polling the communications ports. A given communications port is not currently being used or is open. When a given addressable HA device  31   a - 31   n  communicates via the given communications port not currently being used, the HA security controller  561  determines whether the given addressable HA device is operating abnormally based upon the respective HA device signature and communicates to the cloud  33  for verification of whether the given addressable HA device is operating abnormally. When the given addressable HA device  31   a - 31   n  is verified to be operating abnormally, the HA security controller  561  terminates communications with the given addressable HA device. 
     The HA security controller  561  also generates a notification when the given addressable HA device  31   a - 31   n  is verified to be operating abnormally. In some embodiments, the addressable HA devices  31   a - 31   n  each has a manufacturer associated therewith, and the HA security controller  561  may communicate the notification to a respective manufacturer associated with given addressable HA device verified to be operating abnormally. Of course, the HA security controller  561  may communicate the notification to another device and/or entity, as will be appreciated by those skilled in the art. 
     The HA system  20  may also include a communications interface  563  that provides communication between the HA security controller  561  and the cloud  33 . The HA security controller  561  communicates with the cloud  33  via the communications interface  563 , for example, to update the stored HA device signatures in the memory  562 . 
     The HA system  20  also includes radio controllers  44   a - 44   n  coupled to HA security controller  561 . Each of the addressable devices  31   a - 31   n  may be configured to wirelessly communicate with the HA security controller  561  via respective radio controllers  44   a - 44   n.    
     A method aspect is directed to a method of communicating in the HA system  20 . The method includes using the HA security controller  561  to communicate with the addressable HA devices  31   a - 31   n  via respective ones of the communications ports, with a given communications port not currently being used. The method also includes using the HA security controller  561  to, when a given one of the addressable HA devices  31   a - 31   n  communicates via the given communications port not currently being used, determine whether the given addressable HA device is operating abnormally based upon the respective HA device signature, communicate to the cloud  33  for verification of whether the given addressable HA device is operating abnormally, and terminate communications with the given addressable HA device and generate a notification when the given addressable HA device is verified to be operating abnormally. 
     Another aspect is directed to setup wizards of the K4Home software. A setup wizard may provide an increasingly simple and relatively uniform setup process for each device connected to the K4Connect system  20 , and particularly, connected to K4Home. The setup wizard may limit the actionable items on each screen step of the wizard to maintain simplicity. For example, the setup wizard may allow one question and one data input received from that question before moving to the next step in the wizard. 
     Each setup wizard is based upon prebuilt templates that allow software developers to collect the data for the setting up of a device without having to build new user interface components. Each setup wizard may be customizable for developers and bridge builders, for example. Customization may be achieved by allowing each setup wizard to have a unique style sheet while keeping base styles consistent. Beyond the base styles, the user interface in each setup wizard may be changeable, but it is desirable that these changes be within specific parameters. If no suitable template is available for the developer, for example, user interface components may be created. Custom templates would still use K4Connect components when available and may not contradict the relatively simple and uniform setup process provided by the K4Home software. 
     Referring additionally to  FIGS. 14 a  and 14 b   , each setup wizard may also provide contextual help by supplying a progress bar  71 , for example, on the display  48  of the remote device  36  or as part of the user interface  35 , for example, that includes a help button  72  on the progress bar. The help button  72  links to help that corresponds to the user&#39;s current step in the setup wizard. In other words, the user will be presented with different instructions on the display  48  depending on where the user is in the setup process. This may be particularly advantageous in that it aids users in steps in the setup that are frequently problematic and may make the user experience more adaptive and easier than current home automation setups. 
     Referring now to  FIGS. 15 a -15 c   , the user interface  35  may provide several different ways to control the K4Connect system  20  or to control addressable devices  31   a - 31   n  on the K4Connect system. For example, the K4Connect system  20  may be controlled by room ( FIG. 15 a   ), by scene ( FIG. 15 b   ), and by device types ( FIG. 15 c   ). Of course, the K4Connect system  20  may be controlled in other fashions or using other techniques. 
     The user interface  35 , which may be presented via the display  48  of a remote device  36 , for example, a touch screen display of a mobile phone, allows the user to view addressable devices  31   a - 31   n  by device category or by the location ( FIG. 15 a   ) of the addressable device. The user can also switch directly from the addressable device selection from the location to the addressable device category view. The user interface  35  also advantageously tracks the history of the devices used by tracking the last contacted device. This may allow the user to directly access recently used addressable devices  31   a - 31   n  more quickly instead of searching back through prior pages of the user interface. The user interface  35  may also provide increased usability by allowing the entire screen of the remote device  36 , for example, a touch-screen remote device to be used to adjust the addressable device  31   a - 31   n  instead of locating a single point on the touch-screen display for adjustment. In some embodiments, addressable devices  31   a - 31   n  may be controlled via the user interface  35  by way of voice recognition, for example. Other types of control may also and/or additionally be used, for example, biometrics, or gesture (e.g., arm, hand, eye) recognition. 
     Referring now to  FIG. 16 , when one or more of the addressable devices  31   a - 31   n  are in the form of a light emitting diode (LED) bulb, the user interface  35  includes an LED color picker  75  function. The LED color picker  75  provides a more accurate method to set colors in controllable multi-color LED light bulbs  31   a . Currently, the user selects a color from a palette and the bulb will adjust to closest color possible. This may result in a variation between what the user selects from the display  48  and the actual output from the multi-color LED light bulb  31   a.    
     The LED color picker  75  by way of the processor  49  of the remote device  36 , detects the colors the multi-color LED light bulb  31   a  is capable of producing and presents those color options to the user. This is done, for example, by determining the CIE delta of the multi-color LED light bulb  31   a . The CIE delta may be determined by the manufacturer, the data for which may be stored in the remote device  36  or received from the cloud device  33 . 
     Referring to  FIG. 17 , in another embodiment, when the remote device  36 ′ includes a camera  86 ′, the processor  49 ′ of the remote device may cooperate with the camera to capture the colors actually illuminated by the multi-color LED light bulb  31   a ′. The processor  49 ′ of the remote device  36 ′ then displays on the display  48 ′ the available colors of the multi-color LED light bulb  31   a ′ based upon the stored CIE delta information or the captured images. Colors are calculated in the CIE triangle versus finding the color at the end of the delta. Additionally, in some embodiments, the remote device signature, as discussed above, in the case of a multi-color LED light bulb may include the CIE delta of the bulb based upon the model number, for example. The user then chooses the exact color from the options on the display  48 ′ and the multi-color LED light bulb  31   a ′ changes to selected color. This matches the user expectation to the light bulb output in contrast to the current method which selects a color based upon an approximation. 
     As will be appreciated by those skilled in the art, the capabilities of the multi-color LED light bulb  31   a  are typically much less than what a typical CIE diagram shows. The embodiment described herein advantageously determines the color displaying capabilities of the multi-color LED light bulb  31   a  and allows selection of those actual colors rather than making an approximation. 
     Referring now additionally to  FIG. 18 , the user interface  35  also provides an interface for interacting with multiple K4Hubs hubs or hub devices, for example a home hub  34   a  and an office hub  34   b . Currently in the home automation market, end users either cannot set up multiple hubs in their homes or the hubs are combined in a cloud system preventing the user from being able to make an obvious distinction between the systems. The K4Connect system  20  advantageously permits the user the option of controlling multiple hubs from the user interface  35  by connecting, for example, automatically, to the local hub and connecting to any other hubs through the cloud. 
     When connected to a local network, for example, via Wifi, the user interface  35  of a remote device  36  may automatically connect (i.e., without user intervention) to the hub device  34   a ,  34   b  on the same local network. When using a cellular connection or Wifi network that is not connected to a hub device  34   a ,  34   b , the user interface  35  allows the user to pick which of the multiple systems they would like to view. For example, in a first scenario, a connection to a hub device  34   a  located in the user&#39;s office. The K4App or user interface  35  controls the addressable devices  31   a - 31   n  from the office hub  34   a , but the user has the option to switch the user interface to control other connected hubs. In a second scenario, when the user is connected only to a cellular network such as an LTE network, the user interface  35  provides an option for the user to choose between the connected hubs if there is more than one, so the user can pick between home hub  34   b  or the office hub  34   a . In a third scenario, the user is connected to the home hub  34   b  and the user interface  35  automatically controls the addressable devices  31   a - 31   n  at home, but the user can switch to controlling the office hub  34   a  on the user interface. 
     When a new addressable device  31   a - 31   n  is detected by the home device  32  or the hub device  34  (i.e. a device running K4Home), for example, new software for supporting the newly detected addressable device may be downloaded. For example, an “app store” for controllable devices may provide support or drivers for the newly detected controllable device. The “app store” may be hosted by the cloud server  33  or third party provider, for example. With respect to the app store being available on the cloud server, the cloud server may store in memory addressable device drivers. When a new addressable device  31   a - 31   n  is detected by the home device  32  or hub device  34 , the home or hub device may “pull down” the corresponding driver or software and not an entire software package. 
     Referring now additionally to  FIG. 19 , further details of the bridges  82  will now be described. The K4Connect bridges  82  provide a translation layer for the message queue or message queue server  50  to communicate with the addressable devices  31   a - 31   n  connected to the K4Connect system  20 . When a user or a predefined scene executes a command on the K4Connect system  20 , the message queue  50  sends a generic form of the message through the Node.js APIs to the associated bridge  82 . The generic form of the message may be sent through different APIs or by different techniques as will be appreciated by those skilled in the art. The bridge  82  then translates the generic command to the specific command for the addressable device  31   a - 31   n  and sends the translated command to the addressable device. 
     The independence of the bridges  82  advantageously allows developers to write bridges for nearly any controllable device independently of the whole K4Connect system  20 . After a bridge  82 , which may generally be stored separately from the message queue  50 , is coded, for example, it may be downloaded and integrated into the message queue  50  without having to update the entire K4Connect software program. 
     More particularly, when a new addressable device  31   a - 31   n  is detected by the home device  32  or hub device  34 , for example, new software for supporting the newly detected controllable device may be downloaded, i.e. a bridge. For example, an “app store” for controllable devices may provide support or the bridge for the newly detected controllable device. The “app store” may be hosted by the cloud server  33  or third party provider, for example. With respect to the app store being available on the cloud device  33 , the cloud device may store in memory addressable device bridges. When a new addressable device  31   a - 31   n  is detected by the home device  32  or hub device  34 , the home or hub device may “pull down” the corresponding bridge or software and not an entire software package. 
     The independence of each bridge also allows for better usage of bandwidth and storage space on the home K4Connect system  20 . By not downloading an entire software update package every time a bridge is updated, the user and K4Connect preserve Internet bandwidth and data. Also, the ability to only download the bridges  82  that are desired by each user allows the user to preserve memory space on the device running K4Home, e.g. the home device  32  and/or K4Hub  34 . This preserved memory space allows the K4Connect system  20  to provide a relatively large number of bridges for new home automation devices with less concern of bloated software or limited storage space on user devices, for example. 
     Referring now particularly to  FIG. 20 , when a new bridge  91  is created and loaded to the K4Connect system  20 , the update servers  59  on each K4Connect system connect to the cloud device  33  or K4Away and are notified when the system performs an update. As will be appreciated by those skilled in the art, the update server  59  may perform an update by communicating with the cloud server and determining based upon communication therewith whether an update exists (e.g., based on date, update ID, etc.) The device signature of the new bridge and the device description are sent to the update server  59 . The file or files associated with the device signature and description are generally much smaller than the complete bridge file, which is downloaded if the new controllable device is ultimately connected to the K4Connect system  20 . The update server  59  sends the device signature to the discovery server  35  and the device description to the configuration server  62 . The device signature allows the discovery server  35  to scan available ports and recognize if a new addressable device  31   a - 31   n  that can be connected by the new bridge  91  is in the home. The device description includes the wizard process, for example, as described above, to set up the new controllable device. When the discovery server  55  finds a new controllable device that can be connected by a new bridge  91 , the discovery server  55  sends a message to the configuration server  62  notifying the configuration server of the new addressable device. The discovery server  55  also sends a new addressable device notification to the notification server  66 , which launches the user interface  35  on the display  48  of the remote device  36  to inform the user of the new addressable device. The bridge wizard  92  is also launched. The bridge wizard  92  gathers the information for the device description and requested from the configuration server  62 . 
     Once the information has been gathered and user provides a response, for example via the bridge wizard  92 , the configuration server  62  notifies the loader server  64  of the new configured addressable device. The loader server  64  requests the full bridge download from the update server  59 , and the update server requests the full bridge from the cloud device  33  or K4Away. The update server  59  sends the full bridge download to the loader server  64 , which stores the file and launches the new bridge. The newly connected controllable device is thus connected to the K4Connect system  20 . 
     Referring now additionally to  FIG. 21 a   , the bridges  82   a - 82   c  of the K4Connect system  20  are also what may be referred to by those skilled in the art as “sandboxed” so that the system may be less subject to interruption should a given bridge fails. If one of the bridges  82   a - 82   c  fails or if the connection to the message queue  50  fails, the remaining system components continue to function. The bridges  82   a - 82   c  execute the communication between themselves and the message queue  50  so that if there is a failure in communication, the bridge will generally restart the communication. If a bridge  82   a - 82   c  has an error, for example, the loader server  64  reloads the bridge  82   a - 82   c . These sandboxed processes limit or reduce restarting of the entire software program running on the home device  32  or hub device  34  if an error occurs in a bridge  82   a - 82   c . However, one effect on the K4Connect system  20  may be the inability of controlling the specific addressable devices  31   a - 31   n  associated with the failed bridge  82   a - 82   c , which may be quickly remedied when the loader server  64  reloads the bridge. The functionality of the message queue  50 , other servers, and other bridges are generally unaffected. As noted above, bridges may be installed on demand, for example, as needed, for communicating with addressable HA devices. 
     Referring now to  FIG. 21 b   , the “sandboxed” bridges  82   a - 82   c  will now be described with respect to the HA system  20 . The HA system  20  includes addressable HA devices  31   a - 31   n . The addressable HA devices  31   a - 31   n  may include any of motion detectors, thermostats, light switches, audio controllers, door locks, and/or cameras. Of course, the addressable HA devices  31   a - 31   n  may include other and/or additional devices. The addressable devices  31   a - 31   n  wirelessly communicate using respective different wireless communications protocols from among different wireless communications protocols. 
     A processor  641  and a memory  642  associated with the processor may cooperate to perform the functions described above with respect to the sandboxed bridges  82   a - 82   c . More particularly, the processor  641  and the memory  642  are configured to implement the message queue  50 . That is, the message queue  50  generates generic messages for respective ones of the addressable HA devices  31   a - 31   n . The processor  641  and the memory  642  also implement sandboxed bridges  82   a - 82   c . Each sandboxed bridge  82   a - 82   c  converts a generic message from the message queue  50  into a specific message for a given one of the addressable HA devices  31   a - 31   n . The specific message may be a specific control and/or status message that is specific for the respective sandboxed bridge  82   a - 82   c.    
     Upon failure of one of the sandboxed bridges  82   a - 82   c , the processor  641  and memory  642  implement reloading the failed sandboxed bridge  82   a - 82   c  while maintaining operational the other sandboxed bridges. The processor  641  may determine the failed one sandboxed bridge  82   a - 82   c  based upon communication between the sandboxed bridges and the message queue  50  and/or communication between or among the sandboxed bridges  82   a - 82   c , for example. 
     The HA system  20  also includes radio controllers  44   a - 44   n  coupled to the processor  641 . Each of the addressable devices  31   a - 31   n  may be configured to wirelessly communicate with the processor  641  via respective radio controllers  44   a - 44   n.    
     A method aspect is directed to a method of maintaining operational a plurality of sandboxed bridges  82   a - 82   c  in the HA system  20 . The method includes using the processor  641  and the memory  642  associated therewith to generate, via the message queue  50 , a plurality of generic messages for respective ones of the plurality of addressable HA devices  31   a - 31   n  and convert a generic message from the message queue into a specific message for a given one of the addressable HA devices using the plurality of sandboxed bridges  82   a - 82   c . The method also includes using the processor  641  and memory  642  to, upon a failure of one of the plurality of sandboxed bridges  82   a - 82   c , reload the failed sandboxed bridge while maintaining operational the other sandboxed bridges. 
     Referring now additionally to  FIG. 22 , the K4Home software, which may be executed on the home device  32  or the hub device  34 , also features responsive scenes that function as a list of elements  95  of the K4Connect system  20  that then may induce actions in the addressable devices  31   a - 31   n  connected to the system  20 . The responsive scenes can also return plain language notifications to the user, for example, at the user interface  35 , based on the status of the system  20 . 
     The standard responsive scenes can be set-up by the user by using a scene wizard. The scene wizard includes a list of addressable devices  31   a - 31   n  and command event variables or triggers. The user, for example via the user interface  35  of the K4App, selects the triggers for the scene, the addressable devices  31   a - 31   n  affected, and the actions or states the addressable devices will take to respond to the scene. 
     The standard responsive scene may be initiated by a list of triggers detected by the program or by the user activating the scene  93  in the user interface  35 . An example of a command variable or trigger list is as follows: Trigger 1 is a time period, Trigger 2 is a mobile controlling device being connected to the network, Trigger 3 is a set day, and Trigger 4 is a connected motion detector sensing motion. 
     The scene has specified user defined components or which set of addressable devices  31   a - 31   n  will be contacted for the scene and what state those addressable devices should take. For example, controllable device  1   31   a  is a television (TV), addressable device  2   31   b  is a set of lights in the TV room, addressable device  3   31   c  is a room thermostat in the TV room, and addressable device n  31   n  controls operation of a coffee maker. The system  20  generates a command that is sent to the addressable device. The addressable devices  31   a - 31   n  respond based upon the command. 
     For example, Trigger 1 is activated from 7-9 pm, Trigger 2 is activated when a given user&#39;s smartphone or remote device  36  is connected to the local network, Trigger 3 is activated on weekdays, and Trigger 4 is activated by a living room motion detector detecting motion. Based upon the triggers, the user defined components turn the TV on to a given channel, dim the lights in the TV room, adjust the thermostat to 72 degrees, and begin brewing the evening decaf coffee. 
     The standard responsive scenes may also be shared between users using the cloud device  33  or K4Away, and a marketplace that lists available scenes. The K4Connect system  20  may also suggest possible other and/or additional addressable devices  31   a - 31   n  to connect to add functionality and more responsive scenes to individual users of K4Home. 
     Once a user has completed the responsive scene wizard or has added a shared responsive scene, the remote device  36  via the user interface  35  may display a modeled animation of the scene which shows what the scene looks like upon activation. The user may also access an animation of the scene that will function throughout the entire day and their triggers. 
     Referring now to  FIGS. 23-29   a , another aspect of K4Home is what may be referred to as an ingredient responsive scene based on property based ingredients, which allows the use of different addressable devices that can produce the same properties in the recipes. Instead of a scene being tied to a specific addressable device  31   a - 31   n  for a given function, for example, the scenes are based upon a specific property. This advantageously allows for responsive scenes to be implemented using the same elements that the responsive scene needs, but does not use identical devices. 
     For example, if a given user wants to know when another user is home, they may set up a responsive scene that identifies the addressable devices  31   a - 31   n  that may be used to indicate whether or not someone is home. For the given user, the addressable device  31   a - 31   n  or ingredient in the responsive scene may be a deactivated alarm system, which when tripped gives the desired properties to trigger the responsive scene. The responsive scene then has the K4Connect system  20  send the given user a notification, for example, a plain language notification, that the other user is home. This scene can then be shared with yet a third user who not does not have an alarm system but does have motion detectors  101 , which fall in the same list of devices that can give the desired properties to complete the recipe. In other words, the scene is associated with a desired outcome irrespective of specific addressable devices  31   a - 31   n . In instances where a recipe is almost completed or can be augmented by adding more controllable devices, the K4Connect system  20  informs the user, for example, via the user interface  35  on the remote device  36 , of the possible recipe based responsive scene and links them to an online market where the user can download, either free or for purchase, the addressable device  31   a - 31   n.    
     Another example of a responsive scene based on the ingredients list is if the system  20  indicates that a recipe has not been met, it can then send a plain language notification that the recipe has not been met. For example, if a person has not arrived at home by a certain time, the recipe includes the ingredients of presence (by way of motion detectors, cameras, and a connected smartphone (i.e., remote device  36 )) and time. The lack of presence at a specific time triggers the scene and alerts the user. Of course, others, for example, a monitoring center and/or other designees, may be alerted. 
     A user may set up an ingredient responsive scene (Block  114 ) or download a shared responsive scene from the cloud device  33 . K4Home then determines whether all the ingredients are present in the K4Connect system  20  (Block  102 ). If addressable devices  31   a - 31   n  that can provide ingredients properties are connected the K4Connect system  20 , the system determines the state (Block  106 ) of the addressable devices by polling the addressable devices (Block  103 ). If the all of the ingredients of the scene are met (Block  104 ), then the K4Connect system  20  executes the scene (Block  108 ). If the ingredients of the K4Connect system  20  do not meet the conditions (Block  104 ), then the system may either poll the property states again (Block  103 ) or wait a specified amount of time set by K4Home or the responsive scene. If any of the ingredients/properties are not available in the K4Connect system  20  because an addressable device  31   a - 31   n  that can provide the ingredient is not connected, then K4Home sends a message to the analytics server  54  requesting suggested controllable devices from the cloud device  33  (Block  110 ) and may also cooperate to present the user, for example, on the user interface  35 , an opportunity to purchase the suggested controllable devices ( FIG. 24 ). The new addressable devices may be installed at Block  112 . 
     In another example, a user may download a responsive scene that utilizes a camera to record motion events during a specific time period. For example, a given user wishes to record when his dog climbs onto the living room couch while the given user is at work from 8 am-5 pm. The given user then constructs the scene with three ingredients: ability to record video (provided by a camera connected to K4Home), motion (provided by the same camera&#39;s built in motion detector), and a time period. The given user then shares this on the responsive scene market place on the K4Away or cloud device  33 . Another user downloads the scene and intends to use the scene for home security at night, for example. The other user has a camera, but not the ability to sense motion. K4Home suggests the other user install independent motion sensors to be able to use the scene and provides a link to the K4Store or the cloud device  33  from which the orders of any of a number of brands and styles of motion detectors may be purchased. The other user then installs the motion sensors, which now enables the responsive scene to be enacted since all ingredients are met. The other user then records any motion in his living room from 10 pm-6 am using the same base responsive scene while using different devices to provide the ingredients. 
     For example, a user may generate a responsive scene to provide the idea of “home.” The responsive scene may be generated with respect to the user so that, “when I am home, I want light in the living room.” The K4Connect system  20  indicates or displays, for example, via a menu, that “there are x devices you can use to determine whether I am home, and here are the devices that provide light.” In other words, the scene is constructed first and then the addressable devices  31   a - 31   n  that can make the scene are provided. 
     Referring now to  FIG. 29 b   , ingredient responsive scenes as they relate to the HA system  20  will now be described. The HA system  20  includes addressable HA devices  31   a - 31   n  at a given location. The addressable HA devices  31   a - 31   n  include any of motion detectors, thermostats, light switches, audio controllers, door locks, and/or cameras. Of course, the addressable HA devices  31   a - 31   n  may include additional and/or other devices. 
     The HA system  20  also includes an HA device scene controller  581  that obtains from a user, for example, wirelessly, a first desired scene that includes a first trigger action and first responsive event. For example, the first trigger may be “when I arrive home” and the first responsive event may be “turn on the living room lights”. Indeed, the first trigger action and the first responsive event do not identify which of the addressable HA devices  32   a - 32   n  are responsible for implementing the first trigger action and the first responsive events. The HA device scene controller  581  may obtain the first trigger action and the first responsive event from a user-interface device  360 , for example, and more particularly, a user-input device  351  coupled to a user-interface controller  353  to permit user input. The user interface device  360  may be a remote device, for example, a tablet computer, a smartphone, etc. There may be more than one first trigger action and any number of first responsive events. 
     The HA device scene controller  581  also presents a first user-selectable list of corresponding ones of the addressable HA devices  31   a - 31   n , for example, on a display  354  of the user-interface device  360  coupled to the user-interface controller  353 , that are capable of implementing the first desired scene. In other words, the HA device scene controller  581  presents addressable HA devices  31   a - 31   n  that correspond to or will execute the first trigger action and the first responsive event. 
     The HA device scene controller  581  also determines the first user-selected ones of the addressable HA devices  31   a - 31   n , and upon occurrence of the first trigger event, performs the first responsive event using the first user-selected addressable HA devices to thereby implement the first desired scene. The first desired scene may be executed wirelessly, for example, the HA device scene controller  581  may be communicate wirelessly with the addressable HA devices  31   a - 31   n  to implement the first desired scene. In some embodiments, the HA device scene controller  581  may generate a notification upon occurrence of the trigger event. 
     The HA device scene controller  581  also obtains from the cloud  331 , for example, wirelessly, a second desired scene that includes a second trigger action and second responsive event. The second trigger action and second responsive event are obtained without identifying the addressable HA devices  31   a - 31   n  responsible for implementing the second trigger action and second responsive event. 
     The HA device scene controller  581  may present a second user-selectable list of corresponding addressable HA devices  31   a - 31   n , for example, on the display  354 , capable of implementing the second desired scene. In other words, the second scene is obtained as a shared scene, for example from another person&#39;s HA system. The HA device scene controller  581  also determines the second user-selected addressable HA devices  31   a - 31   n , and, similar to that described above, for example, wirelessly, upon occurrence of the second trigger event, performs the second responsive event using the second user-selected addressable HA devices to thereby implement the second desired scene. 
     The HA device scene controller  581  also may determine when the addressable HA devices  31   a - 31   n  at the given location are not capable of implementing the scene. When this is the case, the HA device scene controller  581  presents a purchase offer, for example on the display  354  for an additional addressable HA device. The user may purchase the additional addressable HA device by clicking on a hyperlink, for example. 
     A method aspect is directed to a method of implementing first and second desired scenes in an HA system  20 . The method includes using an HA device scene controller  581  to obtain from a user the first desired scene that includes a first trigger action and a first responsive event and to present a first user-selectable list of corresponding addressable HA devices  31   a - 31   n  capable of implementing the first desired scene. The HA device scene controller  581  is also used to determine the first user-selected addressable HA devices  31   a - 31   n , and upon occurrence of the first trigger event, perform the first responsive event using the first user-selected addressable HA devices to thereby implement the first desired scene. 
     The HA device scene controller  581  is also used to obtain from the cloud  331  the second desired scene that includes a second trigger action and a second responsive event and to present a second user-selectable list of corresponding addressable HA devices  31   a - 31   n  capable of implementing the second desired scene. The device scene controller  581  is also used to determine the second user-selected ones of the addressable HA devices, and upon occurrence of the at least one second trigger event, perform the second responsive event using the second user-selected addressable HA devices  31   a - 31   n  to thereby implement the second desired scene. In some embodiments, the HA device scene controller  581  is used to determine when the addressable HA devices  31   a - 31   n  at the given location are not capable of implementing the scene, and to present a purchase offer for an additional addressable HA device. 
     Development kits as they relate to the K4Connect system  20  will now be described. The K4Connect system  20  provides both software and hardware development kits. The software development kit builds a complete device stack for developers to interact with and handles all communication with the message queue. A built-in bridge editor allows developers to create and edit bridges from a web browser, for example, and a description editor creates device description XML files. 
     The hardware development kit allows developers to connect controllable devices directly to the message queue without an intermediary bridge. For example, as developers add communication protocols to their controllable devices, the K4Connect system  20 , particularly the communication components thereof, may be integrated into their hardware to bypass a bridge on the system and communicate directly with the message queue. 
     Further details of the cloud device  33  or K4Away will now be described. In addition to the functions of K4Away already described, K4Away hosts an external API, which provides an interface for devices that cannot connect to Internet based services on their own. When connected to the K4Connect system  20  and K4Away, previously un-networked devices may become accessible to outside services such as, for example, IFTTT, Evernote, and Facebook, through its connection with K4Away. 
     With respect to security, the security model of the K4Connect system  20  is based upon providing a relatively high level of security for the system. Each phone or remote device  36  is authenticated on two levels. The first level is a device specific allowance added by the system administrator. The second level is the user login on the remote device  36 . This two-layer system reduces occurrences of a login of unauthorized devices even if there is a valid user login. 
     The K4Connect system  20  also provides security through its privacy method in its analytics data collection. The data is stored on two separate servers. One server holds a token representing the anonymous user while the other server holds the usage and analytic data. The connection between the two servers occurs when authorized by the user for technical help. When the user is sent responsive scene or device recommendations, the suggestions are typically only sent to the token representing the user. The user remains anonymous at all times. In other words, a portion of the information about a user may be selectively available for providing technical support, similar to a “need-to-know” basis. 
     The K4Connect system  20  also uses a security method that grants the user complete rights and ownership to the data collected. The K4Connect system  20  collects and analyzes data from the user and stores it on the separate secure servers. After a threshold time period, for example, one year, the data is permanently deleted. This method includes a user override granting the user the ability to permanently delete their data at anytime. 
     The K4Hub  34  can also be used as a Wifi router connected to a home router so that all the devices connected to the K4Connect system  20  are routed through the private Wifi network of the K4Hub  34 . This advantageously allows for a separation between devices, such as personal computers, connected to the K4Connect system. This separation may reduce the chances of attacks on personal computers from affecting the network among devices of the K4Connect system  20 . 
     Referring now to  FIG. 30 a   , another aspect is directed to health related devices for use on the K4Connect system  20 ″. The use of health related devices in conjunction with the K4Connect system  20 ″ may be termed K4Life. However, it should be noted that other and/or additional devices, whether health related or not, may be part of the K4Life system. Similar to the K4Connect system described above, the K4Life system  20 ″ includes addressable devices  31   a ″- 31   n ″, some of which may be in the form of health devices that measure human health related data, such as, for example, steps walked, blood pressure, weight, and other metrics. In other words, the K4Life system  20 ″ performs the functions of the K4Connect system described above, and includes further health related functions as will be described in further detail below. For example, health devices may include one or more bed sensors, motion detectors, fitness tracking devices, blood pressure cuffs/monitors, weight scales, and temperature probes, for example. Of course, other and/or additional health devices or sensors, for example from the K4Home system, may be used. 
     In addition, the K4App provides social interaction, for example, photo sharing and live video chat. More particularly, when a live video chat is started, the K4Life system  20 ″ may report the start time and duration of the live video chat to a central server, for example, the cloud device  33 ″ or local server device. 
     The K4Life system  20 ″, for example, the analytics server  54 ″, computes a score indicating the overall health of the user, which may be referred as K4Score. The K4Score is determined by combining directly measured health data, activity level measured from the use of addressable devices or health devices  31   a ″- 31   n ″, and social engagement measured by the use of the K4App. The K4Score may include or be based upon other and/or additional information. The historical trend of this score may be used to predict improvement or decline in a user&#39;s health, for example. Of course, this data may be used for other purposes, for example, communicated to other users such as health care professionals, monitoring stations, etc. For example, a person who is sedentary, has irregular sleep patterns, and little social interaction may be identified as a having potential health issues. One example scenario where the K4Life system  20 ″ and K4Score may be relatively advantageous is the use of the system by an elderly parent whose children wish to check on the parent&#39;s wellbeing or if a user simply wants to keep apprised of their own wellbeing. 
     In some embodiments, the health or activity data may be viewed by family members or in a group living setting, such as an assisted living facility or by an onsite or remote supervisor. The health data may also be displayed, for example, via the user interface  35 ″ of the remote device  36 ″, to show the health score of an individual user, or an aggregate of a community of users. 
     Referring now to  FIG. 30 b   , the health related aspects of the K4Connect or HA system  20 ″ will now be described. The HA system  20 ″ includes addressable HA devices  31   a ″- 31   n ″. The addressable HA devices  31   a ″- 31   n ″ may include any of motion detectors, thermostats, light switches, audio controllers, door locks, cameras, and/or health-related sensors (e.g. room occupancy sensors, bed sensors, step counters, heart rate monitors, blood pressure monitors, temperature sensors, and weight scales). Of course, the addressable HA devices  31   a ″- 31   n ″ may include other and/or additional devices. The addressable devices  31   a ″- 31   n ″ wirelessly communicate using respective different wireless communications protocols from among of different wireless communications protocols. 
     The HA system  20 ″ also includes a user interface device  36 ″ that permits user social networking and generates user social networking data based thereon, for example, data related to which social networking applications and an amount of time spent using each social networking application. The user interface device  36 ″ includes a portable housing  361 ″, a display  48 ″ carried by portable housing, wireless communications circuitry  362 ″ carried by the portable housing, and a user interface device controller  49 ″ coupled to the display and wireless communications circuitry for performing at least one wireless communications function. For example, the user interface device  36 ″ may be a smartphone or tablet, and may execute any number of social networking applications, for example, photo sharing, live video chat, and social media applications. 
     The HA system  20 ″ also includes a controller  381 ″ and a memory  382 ″ coupled thereto that stores measured user health data and determines user physical activity data based upon the addressable HA devices  31   a ″- 31   n ″. The physical activity may be determined based upon a period of time period. 
     The controller  381 ″ also generates a user health score based upon the user social networking data, user health data, and user physical activity data, and communicates the user health score via the cloud  331 ″. The controller  381 ″ may also generate user health scores based upon the determined physical activity level at intervals within the period of time, for example. 
     The controller  381 ″ may, for example, generate a notification when the user health score exceeds a threshold. More particularly, if a user health score is indicative of poor health, a notification, such as, for example, an email, SMS message, visual notification on a display, etc. may be generated and communicated to an electronic device  361 ″ via the cloud  331 ″. In some embodiments, the controller  381 ″ may generate a notification if there are consecutive declining user health scores over the time period. Once the user health score is communicated to the cloud  331 ″, it may be downloaded, for example, by the electronic device  361 ″ for storing, viewing, analysis, and/or other data processing as will be appreciated by those skilled in the art. 
     A method aspect is directed to a method of communicating a user health score in the HA system  20 ″. The method includes permitting, via a user interface device  36 ″, user social networking and generating user social networking data based thereon. The method also includes using the controller  361 ″ and the memory  362 ″ coupled thereto to store measured user health data, determine user physical activity data based upon the plurality of addressable HA devices, generate a user health score based upon the user social networking data, user health data, and user physical activity data, and communicate the user health score via the cloud. 
     Referring now to  FIG. 31 , the K4Connect system  20 ′″ may also be used for location determination. The K4Connect system  20 ′″ may detect mobile devices (i.e., remote devices  36 ′″) that are within a specified range of the K4Hub  34 ′″. These detections can be reported to a central server, for example, the cloud device or K4Away where they are used to estimate a person or device&#39;s location within a home or facility, for example. As more than one K4Hub  34 ′″ can detect a mobile or remote device  36 ′″ at a time, the K4Connect system  20 ′″ reduces duplicate data by comparing the detection strength of overlapping data and determining which K4Hub was closest to the detected person or device. Of course, K4Connect system  20 ′″ described in this embodiment may be particularly useful for use with the K4Life system described above. 
     Referring now to  FIG. 32 , in another embodiment, multiple K4Life (or K4Connect) systems  20   a ″″- 20   n ″″ may be used collectively in a system that may be referred to as K4Community. The K4Community system advantageously allows the aggregate data from the multiple K4Life or K4Connect systems  20   a ″″- 20   n ″″, for example, at the cloud device or K4Away, to be analyzed for comparison within the community. Data from other controllers and/or devices may also be aggregated. Of course, any or each system  20   a ″″- 20   n ″″ may process or aggregate the data, for example, entirely or in a shared or load balanced arrangement. Additionally, users in the K4Community  20   a ″″- 20   n ″″ may be able to communicate with each other, and in some embodiments, see how others are performing relative to a given user&#39;s performance. As will be appreciated by those skilled in the art, because health related data is being collected and potentially exchanged, the health related data is maintained anonymous, and may be encrypted, until the user or owner of the health data agrees to share or actually shares it. 
     In some embodiments, the K4Life or K4Community system may not be limited to health related devices and health related data. For example, the principles of the systems described above may be applied to utility management, for example, apartment utility load control management. In such an embodiment, the sensors or controllable devices may be used to monitor energy and water usage, for example, and build a profile based thereon. Particular tenants that use more utilities relative to other tenants may be identified. Common areas may also be monitored and scored. A score may also be assigned to each tenant. 
     Referring to  FIG. 33 , in another embodiment the K4Life system  120  may be used in a health care setting to determine how much time a health care professional is giving a patient or user. In one particular example, the system  120  may be used in a nursing home to monitor how much time a nurse is spending with the user/patient, and when and if the nurse was in the room  147  with the patient. The system  120  and particularly, the hub device  134 , includes a short-range communication protocol controller  199 , such as, for example, Bluetooth. Of course, the hub device  134  may be used interchangeably in this or other embodiments with the home device. Each nurse would also wear an identifying device or tag  197  that includes circuitry  196  configured to communicate with the system via the short-range communication protocol. When the nurse is in the room with the user or patient and is within communication range, the system and tag communicate and the time and duration of communication is logged. This information can be used in a K4Community environment, as will be appreciated by those skilled in the art. 
     Referring now to  FIG. 34 , in another embodiment, for example, in a K4Community system such as a healthcare facility, events or tickets that are based upon addressable devices may be generated. Those events may be logged and/or assigned to staff and displayed on a user interface  135 ′ of a remote device  136 ′. When the staff arrives at the room, for example, of the person associated with the event generation, that staff person&#39;s time of arrival may be logged, for example, as described above. 
     While several embodiments have been described as including software that is executed by a processor or processing circuitry of an electronic device, it should be understood by those skilled in the art, that software may include firmware, machine code, or a configuration of the processors or processing circuitry. Moreover, while several embodiments have been described, it will be appreciated that the functions described in any given embodiment may be used with other and/or additional functions, for example, as described in different embodiments. Still further, while the term “home” has been used to describe certain devices and/or locations (e.g. with respect to home automation), it will be appreciated by those skilled in the art, that the system and its components may be used in other locations, such as apartments, health centers, etc. Thus the term “home” is not specifically limited to a user&#39;s home. Moreover, while a processor and/or controller have been described herein, it will be appreciated that a processor and/or controller may include circuitry for execution respective functions and may also include a memory. A memory may also be coupled to the processor and/or controller, for example. 
     Method aspects include making a home automation integration system as described in any of the embodiments described herein, including K4Connect, K4Life, and K4Community, for example. Other method aspects include operation of the system or the various components thereof as well as performing any of the functions detailed above, for example, integration, communication, display, etc. 
     Another aspect is directed to a non-transitory computer readable medium that stores instructions for executing any of the functions of the systems and methods described herein. For example, the functionality of the K4App, K4Home, and K4Away may be embodied as computer executable instructions stored on a non-transitory computer readable medium. Of course other functions described herein may be embodied on a non-transitory computer readable medium. 
     Referring now to  FIG. 35 , another embodiment is directed to a climate control system  1020  that includes a heating, ventilation, and air conditioning (HVAC) system  1021  for an indoor building area  1022 . The HVAC system  1021  is switchable between operating modes for heating and cooling. The climate control system  1020  includes a home automation (HA) thermostat device  1030  in the indoor building area  1022 . The HA thermostat device  1030  includes a housing  1031  and an indoor temperature sensor  1032  carried by housing. The indoor temperature sensor  1032  senses an indoor temperature of the indoor building area  1022 . 
     A temperature controller  1033  is carried by the housing  1031 . The HA thermostat device  1030  also includes wireless communications circuitry  1034  coupled to the temperature controller  1033 . The wireless communications circuitry  1034  may be configured to communicate via Wifi, cellular, or other protocol, for example. 
     The temperature controller  1033  obtains a setpoint temperature for the indoor building area  1022 . The setpoint temperature may be obtained wirelessly, for example, via the wireless communications circuitry  1034 . The setpoint temperature may be obtained from an input device, a remote electronic device, and/or other device, as will be appreciated by those skilled in the art. 
     The HA thermostat device  1030  also includes a user setpoint temperature input device  1035  and a display  1036 , both carried by the housing  1031  and coupled to the temperature controller  1033 . The user setpoint temperature input device  1035  may be in the form of a touch display, pushbutton, rotatable dial, or other input device, as will be appreciated by those skilled in the art. The user setpoint temperature input device  1035  may be used to set the setpoint temperature. The temperature controller  1033  may cooperate with the display  1036  to display the indoor temperature and the setpoint temperature. 
     The setpoint temperature may also be generated or set based upon an HA controller  1037 , for example, as described above, and coupled to the HA thermostat device  1030  and configured to generate the setpoint temperature. As described above, the HA controller  1037  may be coupled to addressable HA devices  1038   a - 1038   n , for example, motion detectors, lighting, etc. The HA controller  1037  generates the setpoint temperature based upon one of the addressable HA devices  1038   a - 1038   n . For example, based upon motion detected from a motion detector, the HA controller  1037  may communicate with the HA thermostat device  1030  to set the setpoint temperature (i.e., set the setpoint temperature cooler when someone is home). Of course, the setpoint temperature can be set based upon other types of addressable HA devices  1038   a - 1038   n.    
     The temperature controller  1033  also obtains an external temperature from external to the indoor building area  1022 . The external temperature may be obtained wirelessly, for example, via the Internet. The external temperature may be an outside temperature or may be an inside temperature of a room or area that may be considered external to the indoor building area  1022 , for example. In some embodiments, more than one temperature sensor (indoor and/or outdoor) may be used to obtain the external temperature. 
     The temperature controller  1033  determines a crossing of the external temperature of the setpoint temperature, and switches the HVAC system  1021  between operating modes based upon the crossing of the external temperature of the setpoint temperature and the indoor temperature moving beyond the setpoint temperature by a threshold temperature difference, for example, one degree. Other threshold temperature differences may be used. 
     Referring now additionally  FIG. 36 , operation of the climate control system  1020  is illustrated by way of the graph  1040  and corresponding displays  1036   a - 1036   e  that show corresponding indoor temperatures  1043   a - 1043   e  and setpoint temperatures  1044   a - 1044   e  at different points in time identified on the graph. In the graph  1040 , the outside temperature is shown by the line  1041 , while the actual or indoor temperature is shown by the line  1042 . Illustratively, the indoor temperature or room temperature deviates from desired temperature or setpoint temperature momentarily while the external temperature passes through the deadbands  1045 . 
     A method aspect is directed to a method of operating the climate control system  1020 . The method includes sensing the indoor temperature of the indoor building area  1022  via the indoor temperature sensor  1032 . The method also includes using the HA thermostat device  1030  in the indoor building area  1022  to obtain a setpoint temperature for the indoor building area, obtain an external temperature from external to the indoor building area, determine a crossing of the external temperature of the setpoint temperature, and switch the HVAC system  1021  between operating modes based upon the crossing of the external temperature of the setpoint temperature and an indoor temperature of the indoor building area  1022  moving beyond the setpoint temperature by a threshold temperature difference. 
     Referring now to  FIGS. 37 a -37 e   , in another embodiment of an HA system  2020 , it may be desirable to remotely access the addressable HA devices  2031   a - 2031   n . Remote access of the addressable HA devices  2031   a - 2031   n , also known as IOT devices, may be particularly helpful for troubleshooting an issue with a given addressable HA device and/or updating software or a configuration, for example. 
     The addressable HA devices  2031   a - 2031   n  are typically behind one or more a network address translation (NAT) routers and/or firewalls, and are thus not generally internet accessible, as will be appreciated by those skilled in the art. Accordingly, to access the addressable HA devices  2031   a - 2031   n , on-demand secure shell (SSH) tunneling may be used. 
     On-demand SSH tunneling allows a given one of the addressable HA devices  2031   a - 2031   n  to communicate with, for example, through periodic connections, a known host to retrieve tunneling instructions. The tunneling instructions may thus permit remote access to a given addressable HA device with reduced overhead through the SSH protocol, for example. Of course, other protocols, for example, secure protocols, may be used. 
     To establish a remote connection to an addressable HA or IOT device  2031   a - 2031   n  a request is issued for a given addressable HA device to open a tunnel. This may be performed using on-demand SSH tunneling by way of a remote user, for example, via remote access wireless communications device  2036 , creating a file in a web-visible (e.g. publically accessible) location that is specific to the given addressable HA device ( FIG. 37 a   ). The web-visible location may be on a server  2099  or other web-visible location, for example. In an example embodiment, the file may be an Amazon simple storage service (S3) file that is a hash of the given addressable HA device&#39;s unique identification and the last unique cloud session identification. The S3 file may include other and/or additional information about the addressable HA device  2031   a - 2031   n.    
     A cloud server  2033  may make available the device specific instructions for the given HA device  2031   a - 2031   n  in a known location, for example, on the server  2099  ( FIG. 37 b   ). In some embodiments, the device specific instructions may be collocated on the cloud server  2033 . 
     Addressable HA devices  2031   a - 2031   n  communicate with, for example, by periodically polling, this location. For example, the location may be polled every few minutes. Of course, the addressable HA devices  2031   a - 2031   n  may communicate with or poll the location at longer, shorter, and/or different intervals. Based upon the polling, for example, the addressable HA device  2031   a - 2031   n  finds the tunneling instructions stored and made available for the given addressable HA device ( FIG. 37 c   ). In one example, the instructions may be a json file that includes a cloud-visible host/port/username/password. Of course, the instructions may be embodied in a different type of file and/or other data elements may be stored in the instruction file. 
     The given addressable HA device  2031   a - 2031   n  opens an SSH tunnel to the cloud server  2033  according to the instructions retrieved from the web-accessible location ( FIG. 37 d   ). The remote user via the remote access wireless communications device  2036 , may then connect to the cloud-end of the tunnel permitting communication with, for example, by way of logging into, the given addressable HA device  2031   a - 2031   n  as though it were internet-visible ( FIG. 37 e   ). 
     Referring now to  FIGS. 38 and 39 , in another embodiment, a home automation (HA) system  3020  includes HA operation devices  3031   a - 3031   n  within a user living area  3021 , for example, a senior living facility. Similar to the HA operation devices described above, the HA operation devices  3031   a - 3031   n  may include IoT devices, such as, for example, light switches, motion detectors, bed occupancy sensors, cameras, door locks, thermostats, medication containers, etc. 
     The HA system  3020  also includes an HA hub device  3034  to provide communications for the HA operation devices  3031   a - 3031   n . The HA hub device  3034  operates similar to embodiments of the HA hub devices described above, and thus, need not be further described. 
     The HA system  3020  also includes a HA user interface  3036  that wirelessly communicates with one or more of the HA operation devices  3031 - 3031   n . The HA user interface device  3036  may be a user interface tablet computer, for example. Of course, the HA user interface device  3036  may be another type of device, for example, a desktop computer, smartphone, etc., and there may be more than one HA user interface device. The HA user interface device  3036  operates similarly to embodiments of the HA user interface devices described above, and thus, need not be further described. 
     A controller  3037  is illustratively carried by the HA hub device  3034 . The controller  3037  may include a processor or other circuitry and may perform functions associated with the HA hub device  3034 , for example, as described with respect to the embodiments above. 
     Referring now additionally to the flowchart  3060  in  FIG. 40 , beginning at Block  3062 , the controller  3037  stores, for example, in a memory  3038  coupled to the controller, historical operational data for each of the HA operation devices  3031   a - 3031   n  based upon a user within the user living area  3021  (Block  3064 ). The historical operational data may include data indicative of whether and when the user has operated and/or activated the HA operation devices  3031   a - 3031   n . The historical operational data may include periods of non-operation and/or inactivity of the HA operation devices  3031   a - 3031   n.    
     At Block  3066 , the controller  3037  uses machine learning to determine a predicted operational pattern of one or more of HA operation devices  3031   a - 3031   n  based upon the stored historical operational data. More particularly, the controller  3037 , based upon the historical operational data, may predict that the user may operate and/or activate one or more of the HA operation devices  3031   a - 3031   n  in a specified manner and at, or within a threshold of, a specified time. In one implementation example where HA operation devices  3031   a - 3031   n  include a bed occupancy sensor and a motion sensor, the controller  3037  may predict that the user rests or sleeps in the bed daily between the hours of 10 pm and 7 am, and that the user wakes from the bed to use the restroom typically daily at around 4 am, for example, based upon the occupancy sensor. Of course, the controller  3037  may make a prediction for a single HA operation device  3031   a - 3031   n  or more than two HA operation devices. 
     At Block  3068 , the controller  3037  monitors operation of the HA operation devices  3031   a - 3031   n  and determines therefrom an HA operation device deviation from the predicted operational pattern (Block  3070 ). For example, with reference to the above implementation example, if the user is not in bed by 11 pm or midnight, and/or if the user does not wake at about 4 am to use the restroom, the controller  3037  may determine that there is an HA operation device deviation from the predicted operational pattern. The controller  3037  may determine an HA operation device deviation from the predicted operational pattern if the predicted operational pattern does not occur within a threshold time period from the predicted operational pattern. With respect to the implementation example described above, the controller  3037  may determine an HA operation device deviation if the user is not in bed by 11 pm (i.e., a one hour grace period from the predicted operational pattern) and does not wake to use the restroom between 3:30 am and 4:30 am (i.e., a half-hour deviation before and after the predicted operational pattern). More than one or other HA operation devices  3031   a - 3031   n  may be used by the controller  3037  to determine an HA operation device deviation with respect to a given HA operation device. 
     If, at Block  3070 , an HA operation device deviation from the predicted operational pattern is determined, the controller  3037  generates a notification  3029  based upon the determined HA operation device deviation (Block  3072 ). The notification may be in the form of an email, SMS message, visual notification, and/or audible notification. In one implementation example where the living area  3021  is a senior living facility, the notification may be in the form of an audible and visual alarm at a remote computer at a community staff station or similar monitoring location. The notification may indicate that a check-in with the user should take place as the user may have fallen, not returned to the living area  3021 , or there being another health related issue. If, at Block  3070 , an HA operation device deviation from the predicted operational pattern is not determined, the controller  3037  will continue to monitor operation of the HA operation devices  3031   a - 3031   n , which may be stored as historical operational data, and use machine learning to update the predicted operational pattern. 
     As noted above, the controller  3037  uses machine learning to predict an operational pattern of one or more of the HA operation devices  3031   a - 3031   n . The controller  3037  may learn the predicted operational pattern based upon supervised linear regression and/or recurrent neural networks. Other and/or additional deep learning algorithms may be used. As will be appreciated by those skilled in the art, both supervised linear regression and recurrent neural networks algorithms involve labeling of the data. Based upon the generated notifications, with respect to the implementation example, where the living area  3021  is a senior living center, community staff members will typically take an action in response to the notification  3029 . 
     In some embodiments, these responsive actions, for example, by the community staff members, are provided, as input, to the controller  3037 , and more particularly, to models for machine learning to validate whether a given HA operation device deviation (e.g., whether the user was not in bed, and/or suffering a health issue. The labeling of the data helps train the models used to determine the predicted operational pattern. Once the labeling is done, the data may be split into two parts: a training set—used to train the models and a testing set—used to test a model/hypothesis. The controller  3037  adjusts the models, for example, based upon the responsive action input, until a desired accuracy is achieved. In some embodiments, the controller  3037  may implement or execute different algorithms to determine which algorithm provides higher accuracy results for a given set of data. The controller  3037  may also daisy chain a series of algorithms together to achieve a desired accuracy, for example, greater than 70%, or more preferably, greater than 80%. 
     Once a desired accuracy is achieved, the actual data (i.e., from monitoring the operation of the HA operation devices  3031   a - 3031   n  and the historical operational data) serves the basis for determining the HA operational device deviation, and thus, generating of the notification. Thus “fake” alerts or notifications may be reduced and productivity may be improved. More particularly, a notification may be considered “fake” or “genuine” based upon a threshold, for example, of desired accuracy. Newly collected or stored operational data, for example, from monitoring operations, may be used to further reinforce learning so that the models improve with the amount of data stored, collected, and notifications generated. Operations end at Block  3074 . 
     Referring now to  FIG. 41 , in another embodiment, the controller  3037 ′ may also determine an HA user device deviation based upon operational data from the HA user interface device  3036 ′. More particularly, the controller  3037 ′ stores historical operational data for the HA user interface device  3036 ′ (or more than one HA user interface device, for example, associated with the user), uses machine learning to determine a predicted operational pattern of the HA user interface device based upon the stored historical operational data, and monitors operation of the HA user interface device and determines therefrom an HA user interface device deviation from the predicted operational pattern. Based upon the determined HA user interface device deviation, the controller  3037 ′ may generate a notification. As will be appreciated by those skilled in the art, the controller  3037 ′ may determine an HA user device deviation based upon operational data from the HA user interface device  3036 ′ independently from the HA operation devices  3031   a ′- 3031   n ′ and/or in combination with the HA operation devices. In one implementation example, the controller  3037 ′ may determine that the user typically uses their HA user interface device  3036 ′ (e.g., a tablet) for about 2 hours a day and/or within a certain time period (e.g., 2-4 pm). If the user does not reach 2 hours a day or within a threshold thereof or during the typical time period, or within a threshold thereof, the controller  3037 ′ may determine an HA user device deviation and generate a notification based upon the determined HA user interface device deviation. Other functions of the controller  3037 ′ not specifically described are similar to those described above with respect to determining an HA operation device deviation based upon operational data from the HA user operation devices  3031   a ′- 3031   n′.    
     Referring now to  FIG. 42 , in another embodiment, the controller  3037 ″ is carried by the HA user interface device  3036 ″. In other words, in addition to the functions described above with respect to determining an HA user device deviation, the controller  3037 ″, in the present embodiment, performs operations associated with the HA user interface device  3036 ″, for example, as described above. 
     Referring briefly to  FIG. 43 , in another embodiment, the HA system  3020 ′″ may include multiple controllers  3037   a ′″,  3037   b ′″. One controller  3037   a ′″ may be carried by the HA hub device  3034 ′″, and one controller  3037   b ′″ may be carried by the HA user interface device  3036 ′″. Thus, the controllers  3037   a ′″,  3037   b ′″, in addition to performing their respective device functions, may split or perform some of the controller functions described above with respect to determining HA device deviations based upon respective operational data. More than one controller may be carried by either or both of the HA user interface device  3036 ′″ and the HA hub device  3034 ′″. 
     Referring now to  FIG. 44 , the HA system  3020 ″″ may also include a cloud server  3033 ″″ remote from the HA hub device  3034 ″″ in a cloud computing environment  3099 ″″ and carrying the controller  3037 ″″. In other words, the controller  3037 ″″ may perform any of the functions described herein with respect to determining HA device deviations based upon respective operational data, while not necessarily performing respective device functions, such as, for example, the functions of the HA operation devices  3031   a ″″- 3031   n ″″, the HA user interface device  3036 ″″, and the HA hub device  3034 ″″. Of course, in some embodiments, the controller  3037 ″″ in the cloud computing environment  3099 ″″ may perform at least some functions of the HA operation devices  3031   a ″″- 3031   n ″″, the HA user interface device  3036 ″″, and/or the HA hub device  3034 ″″. 
     Referring now briefly to  FIG. 45 , in another embodiment, the HA system  3020 ′″″ may include multiple controllers  3037   a ′″″,  3037   b ′−″. One controller  3037   a ′″″ may be carried by the HA hub device  3034 ′″″, and one controller  3037   b ′″″ may be carried by the cloud server  3033 ′″″ in the cloud computing environment  3099 ′″″. More than one controller may be carried by either or both of the cloud server  3033 ′″″ and the HA hub device  3034 ′″″. 
     A method aspect is directed to a method of monitoring operation of a plurality of home automation (HA) operation devices  3031   a - 3031   n  of an HA system  3020  within a user living area  3021 . The home automation (HA) system  3020  includes an HA hub device  3034  to provide communications for the plurality of HA operation devices  3031   a - 3031   n . The method also includes using at least one controller  3037  to store historical operational data for each of the plurality of HA operation devices  3031   a - 3031   n  based upon a user within the user living area  3021 , and use machine learning to determine a predicted operational pattern of at least one of the plurality of HA operation devices based upon the stored historical operational data. The method also includes using the at least controller  3037  to monitor operation of the plurality of HA operation devices  3031   a - 3031   n  and determine therefrom an HA operation device deviation from the predicted operational pattern, and generate a notification  3029  based upon the determined HA operation device deviation. 
     A computer readable medium aspect is directed to a non-transitory computer readable medium for monitoring operation of a plurality of home automation (HA) operation devices  3031   a - 3031   n  of an HA system  3020  within a user living area  3021 . The HA system  3020  includes an HA hub device  3034  to provide communications for the plurality of HA operation devices  3031   a - 3031   n . The non-transitory computer readable medium includes computer executable instructions that when executed by at least one controller  3037  cause the at least one controller to perform operations. The operations include storing historical operational data for each of the plurality of HA operation devices  3031   a - 3031   n  based upon a user within the user living area  3021  and using machine learning to determine a predicted operational pattern of at least one of the plurality of HA operation devices based upon the stored historical operational data. The operations also include monitoring operation of the plurality of HA operation devices  3031   a - 3031   n  and determine therefrom an HA operation device deviation from the predicted operational pattern, and generating a notification  3029  based upon the determined HA operation device deviation. 
     Referring now to  FIG. 46 , in another embodiment, a home automation (HA) system  4020  includes HA operation devices  4031   a - 4031   n  within a user living area  4021 , for example, a senior living facility. More particularly, at least one of the HA operation devices  4031   a - 4031   n  is within a restroom  4022  in the user living area  4021 . Similar to the HA operation devices described above, the HA operation devices  4031   a - 4031   n  may include IoT devices, such as, for example, light switches, motion detectors, bed occupancy sensors, cameras, door locks, thermostats, medication containers, toilet sensors (e.g., flush sensors carried by toilets, seat cover open/closed sensors, weight-on-seat sensors) etc. 
     The HA system  4020  also includes an HA hub device  4034  to provide communications for the HA operation devices  4031   a - 4031   n . The HA hub device  4034  operates similar to embodiments of the HA hub devices described above, and thus, need not be further described. 
     The HA system  4020  also includes an HA user interface device  4036  that wirelessly communicates with one or more of the HA operation devices  4031 - 4031   n . The HA user interface device  4036  may be a user interface tablet computer, for example. Of course, the HA user interface device  4036  may be another type of device, for example, a desktop computer, smartphone, etc., and there may be more than one HA user interface device. The HA user interface device  4036  operates similarly to embodiments of the HA user interface devices described above, and thus, need not be further described. In some embodiments, the HA system  4020  may not include an HA user interface device  4036 . 
     A controller  4037  is illustratively carried by the HA hub device  4034 . The controller  4037  may include a processor or other circuitry and may perform functions associated with the HA hub device  4034 , for example, as described with respect to the embodiments above. 
     Referring now additionally to the flowchart  4060  in  FIG. 47 , beginning at Block  4062 , the controller  4037  stores, for example, in a memory  4038  coupled to the controller, historical operational data for the HA operation devices  4031   a - 4031   n  within the restroom  4022  based upon a user within the restroom (Block  4064 ). The historical operational data may include data indicative of whether and when the user has operated and/or activated the HA operation devices  4031   a - 4031   n  within the restroom  4022 . The historical operational data may include periods of non-operation and/or inactivity of the HA operation devices  4031   a - 4031   n  within the restroom  4022 . In some embodiments, the controller  4037  may store historical operational data for other HA operation devices  4031   a - 4031   n  that are not within the restroom  4022 . For example, with respect to a given implementation example, the user may historically use the restroom (e.g., as determined by a toilet seat sensor, flush sensor, and/or motion sensor in the restroom  4022 ), five times a day: upon waking, once mid-morning, after lunch, after dinner, and right before bedtime and for about 5 minutes for visit. 
     At Block  4066 , the controller  4037  monitors operation of the HA operation devices  4031   a - 4031   n  within the restroom  4022  and determines therefrom whether current usage of the restroom (e.g., the toilet) has changed based upon the historical operational data (Block  4068 ). More particularly, the controller  4037  may determine that the frequency of usage of the restroom  4022  has changed (e.g., increased) and/or that the duration of usage has also changed (e.g., increased) based upon the historical operational data. For example, with reference to the above implementation example, the controller  4037  would determine if the user&#39;s usage of the restroom  4022  has increased, from five times a day to ten times a day and/or the user&#39;s duration in the restroom has increased from 5 minutes per visit to 15-20 per visit. The controller  4037  may also determine the change as a decrease in usage (e.g., in frequency and/or duration). The controller  4037  may determine that the current usage has changed if the usage changes (e.g., increased or decreases) by a threshold amount, for example, twenty-five percent. Of course, other thresholds may be indicative of a change in current usage (e.g., one use). Thresholds indicative of a change in current usage may be set, for example, on a user-by-user basis. Moreover, other HA operation devices  4031   a - 4031   n  that may not be within the restroom  4022  may be used to assist in determining the change in current usage. For example, if the user gets out of bed in the 3 am hour, and thereafter proceeds to use the restroom  4022  as sensed by a motion sensor within the restroom, this may serve as confirmation of increased usage, as the user did not likely get out of bed to perform other actions, such as shower, bathe, etc. With respect to the implementation example described above, the controller  4037  may determine that the user&#39;s current usage of the restroom  4022  increased to about twice what has typically been considered normal (e.g., between 9-11 times a day versus about 5 times a day historically) and the user currently spends about 15-20 minutes per restroom visit as opposed to 5 minutes per visit. The change in the current usage, in either or both of duration and frequency, may be indicative of a health issue. For example, an increase in duration of a bathroom visit may be indicative that the user has fallen. 
     In some embodiments, the controller  4037  may use a machine learning based model to analyze various factors to consider if the user&#39;s frequency of restroom usage is less than normal irrespective of time of the day. One such example could be seasonal variation or variation due to diet consumed during dinner/lunch. The controller  4037  or may consider “crowd data” or data from other users within the living area  4021  or senior living facility who may share a similar demography (e.g., age, gender, health problems, etc.) and consider if the frequency of restroom visits are not normal. 
     If, at Block  4068 , current usage of the restroom  4022  for the given user is determined to have changed (e.g., increased), the controller  4037  generates a notification based upon the historical operational data and the determined current usage of the restroom (Block  4070 ). The notification may be in the form of an email, SMS message, visual notification, and/or audible notification. In one implementation example where the living area  4021  is a senior living facility, the notification may be in the form of an audible and visual alarm at a remote computer at a community staff station or similar monitoring location. The notification  4029  may indicate that a medical status of the user should be inquired as the changed usage of restroom  4022  may be indicative of a medical condition needing attention, for example, and for that the user may be unaware. If, at Block  4068 , a change in current usage of the restroom  4022  is not determined, the controller  4037  continues to monitor operation of the HA operation devices  4031   a - 4031   n  within the restroom  4022 , which may be stored as historical operational data. Operations end at Block  4072 . 
     Referring now to  FIG. 48 , in another embodiment, the controller  4037 ′ is carried by the HA user interface device  4036 ′. In other words, in addition to the functions described above with respect to determining an HA user device deviation, the controller  4037 ′, in the present embodiment, performs operations associated with the HA user interface device  4036 ′, for example, as described above. 
     Referring briefly to  FIG. 49 , in another embodiment, the HA system  4020 ″ may include multiple controllers  4037   a ″,  4037   b ″. One controller  4037   a ″ may be carried by the HA hub device  4034 ′″, and one controller  4037   b ″ may be carried by the HA user interface device  4036 ″. Thus, the controllers  4037   a ″,  4037   b ″, in addition to performing their respective device functions, may split or perform some of the controller functions described above with respect to determining HA device deviations based upon respective operational data. More than one controller may be carried by either or both of the HA user interface device  4036 ″ and the HA hub device  4034 ″. 
     Referring now to  FIG. 50 , the HA system  4020 ′″ may also include a cloud server  4033 ′″ remote from the HA hub device  4034 ′″ in a cloud computing environment  4099 ′″ and carrying the controller  4037 ′″. In other words, the controller  4037 ′″ may perform any of the functions described herein with respect to determining HA device deviations based upon respective operational data, while not necessarily performing respective device functions, such as, for example, the functions of the HA operation devices  4031   a ′″- 4031   n ′″, the HA user interface device  4036 ′″, and the HA hub device  4034 ′″. Of course, in some embodiments, the controller  4037 ′″ in the cloud computing environment  4099 ′″ may perform at least some functions of the HA operation devices  4031   a ′″- 4031   n ′″, the HA user interface device  4036 ′″, and/or the HA hub device  4034 ′″. 
     Referring now briefly to  FIG. 51 , in another embodiment, the HA system  4020 ″″ may include multiple controllers  4037   a ″″,  4037   b ″″. One controller  4037   a ″″ may be carried by the HA hub device  4034 ′″″, and one controller  4037   b ″″ may be carried by the cloud server  3033 ′″″ in the cloud computing environment  4099 ″″. More than one controller may be carried by either or both of the cloud server  4033 ″″ and the HA hub device  4034 ″″. 
     A method aspect is directed to a method of monitoring operation of at least one HA operation device  4031   a - 4031   n  of an HA system  4020  and within a restroom  4022  of a user living area  4021 . The HA system  4020  includes an HA hub device  4034  to provide communications for the at least one HA operation device  4031   a - 4031   n . The method includes using at least one controller  4037  to store historical operational data for the at least one HA operation device  4031   a - 4031   n  based upon a user within the restroom  4022 , and monitor operation of the at least one HA operation device, and determine therefrom whether current usage of the restroom has changed based upon the historical operational data. The controller  4037  is also used to generate a notification based upon the historical operational data and the determined current usage of the restroom  4022 . 
     A computer readable medium aspect is directed to a non-transitory computer readable medium for monitoring operation of at least one HA operation device  4031   a - 4031   n  of an HA system  4020  and within a restroom  4022  of a user living area  4021 . The HA system  4020  includes an HA hub device  4034  to provide communications for the at least one HA operation device  4031   a - 4031   n . The non-transitory computer readable medium includes computer executable instruction that when executed by at least one controller  4037  causes the at least one controller to perform operations. The operations include storing historical operational data for the at least one HA operation device  4031   a - 4031   n  based upon a user within the restroom  4022 , and monitoring operation of the at least one HA operation device, and determine therefrom whether a frequency of usage of the restroom has increased based upon the historical operational data. The operations also include generating a notification based upon the determined increased frequency of usage of the restroom  4022 . 
     Referring now to  FIG. 52 , in another embodiment, an HA system  5020  includes HA operation devices  5031   a - 5031   n  that may be within a user living area  5021 , for example, a senior living facility. Similar to the HA operation devices described above, the HA operation devices  5031   a - 5031   n  may include IoT devices, such as, for example, light switches, motion detectors, bed occupancy sensors, cameras, door locks, toilet sensors, thermostats, medication containers, etc. The HA operation devices  5031   a - 5031   n  may also include fitness trackers, such as, for example, accelerometer-based devices to track steps (i.e., pedometer), movement, or other activity. 
     The HA system  5020  also includes an HA hub device  5034  to provide communications for the HA operation devices  5031   a - 5031   n . The HA hub device  5034  operates similar to embodiments of the HA hub devices described above, and thus, need not be further described. 
     The HA system  5020  also includes a HA user interface device  5036  that wirelessly communicates with one or more of the HA operation devices  5031 - 5031   n . The HA user interface device  5036  may be a user interface tablet computer, for example. Of course, the HA user interface device  5036  may be another type of device, for example, a desktop computer, smartphone, etc., and there may be more than one HA user interface device. The HA user interface device  5036  operates similarly to embodiments of the HA user interface devices described above, and thus, need not be further described. 
     The HA system  5020  also includes a biometric sensor  5059 . The biometric sensor  5059  may be in the form of a pulse sensor, pulse oximetry sensor, blood pressure sensor, heart rate sensor, and/or a weight sensor. Of course, the biometric sensor  5059  may sense other and/or additional biometrics, also sometimes referred to as a person&#39;s vital signs. 
     A controller  5037  is illustratively carried by or within the HA hub device  5034 . The controller  5037  may include a processor or other circuitry and may perform functions associated with the HA hub device  5034 , for example, as described with respect to the embodiments above. The controller  5037  may be carried by other and/or additional devices, for example, the HA user interface device  5036 . The controller  5037  may be carried by another device or devices, or embodied in another device or devices, for example, as will be described in further detail below. 
     Referring now additionally to the flowchart  5060  in  FIG. 53 , beginning at Block  5062 , the controller  5037  cooperates with the biometric sensor  5059  to monitor a biometric characteristic of a user (Block  5064 ). The biometric characteristic may include any one or more of a weight of the user, blood pressure of the user, heart rate of the user, and blood-oxygen level of the user. The biometric characteristics may include other and/or additional biometric characteristics, as will be appreciated by those skilled in the art. 
     The controller  5037  stores, for example, in a memory  5038  coupled to the controller, historical operational data for at least one of the HA operation devices  5031   a - 5031   n  based upon the user, for example, within the user living area  5021  (Block  5066 ). The historical operational data may include data indicative of whether and when the user has operated and/or activated the HA operation devices  5031   a - 5031   n . The historical operational data may include periods of non-operation and/or inactivity of the HA operation devices  5031   a - 5031   n . The historical operational data may include quantitative values from an HA operation device  5031   a - 5031   n , for example, a user&#39;s number of steps on a given day, awake hours in a given day, etc. 
     In some embodiments, the controller  5037  may store the historical operational data for one or more of the HA operation devices  5031   a - 5031   n  based upon another user or users. The other user or users may be within the user living area  5021 , for example. 
     At Block  5068 , the controller  5037  determines a data trend of the HA operation devices  5031   a - 5031   n  for which there is stored historical operational data (e.g., which may include historical operational data for the user and/or another user or users). For example, in the case of the number of steps a user has taken in a given day, the controller  5037  may determine whether, based upon the historical operational data, the user is becoming more active or less active. The controller  5037  may make this determination based upon other and/or additional HA operation devices  5031   a - 5031   n , for example, based upon light switch activity and bed sensor data, which may indicate whether, in conjunction, with the steps walked, whether the user is more or less active, and at which times. 
     The controller  5037  correlates, for example, based upon time, the data trend with the biometric characteristic data of the user (Block  5070 ). More particularly, with respect to the data trend of a user becoming less active, the controller  5037  may correlate the biometric characteristic of the user, e.g., at those given times, in an attempt to see if there is a correlation between the biometric characteristic that can be attributed to the decrease in activity (e.g., blood pressure, blood sugar levels, etc.). 
     At Block  5072 , the controller  5037  uses machine learning to predict a health change of the user based upon the correlated data trend and biometric characteristic of the user. For example, with respect to the above example, using machine learning, the controller  5037  may predict the onset of a medical condition based upon the trend of decreased activity and correlated blood pressure and blood sugar. 
     At Block  5074 , the controller  5037  may optionally generate a notification of the health change and communicate the notification, for example, to the user, guardian, and/or health care professional. The notification may be in the form of an email, SMS message, visual notification, and/or audible notification. In one implementation example where the living area  5021  is a senior living facility, the notification may be in the form of an audible and visual alarm at a remote computer at a community staff station or similar monitoring location. 
     As noted above, the controller  5037  uses machine learning to predict a health change of the user based upon the correlated data trend and biometric characteristic of the user. The controller  5037  may learn the predicted health changes based upon supervised linear regression and/or recurrent neural networks. Other and/or additional deep learning algorithms may be used. As will be appreciated by those skilled in the art, both supervised linear regression and recurrent neural networks algorithms involve labeling of the data. In some embodiments, e.g., where the historical operational data for one or more of the HA operation devices  5031   a - 5031   n  based upon another user or users is stored, the health change may be predicted based upon, additionally or alternatively, the correlated data trend for the given user to a data trend of other users. 
     A notification generated by the controller  5037  may be acted upon, for example, by a medical professional. In some embodiments, these responsive actions may be provided, as input, to the controller  5037 , and more particularly, to models for machine learning to validate the predicted health change. The labeling of the data helps train the models used to determine the predicted health change. Once the labeling is done, the data may be split into two parts: a training set—used to train the models and a testing set—used to test a model/hypothesis. The controller  5037  adjusts the models, for example, based upon the responsive input or validation, until a desired accuracy is achieved. In some embodiments, the controller  5037  may implement or execute different algorithms to determine which algorithm provides higher accuracy results for a given set of data. The controller  5037  may also daisy chain a series of algorithms together to achieve a desired accuracy, for example, greater than 70%, or more preferably, greater than 80%. 
     Once a desired accuracy is achieved, the actual data may serve the basis for determining the health change. Newly collected or stored operational data and biometric characteristic data may be used to further reinforce learning so that the models improve with the amount of data stored, collected, and notifications generated. 
     Additionally, in embodiments where historical operational data for one or more of the HA operation devices  5031   a - 5031   n  is considered (i.e., historical volume/crowd data), a given user&#39;s physical activities, mood swings, behavior, or other symptoms may serve as a basis for comparing the given user&#39;s individual data pattern against historical data from other users. Advantageously, it may be determined that the given user has a relatively high risk of a chronic disease, for example, and a timely prediction may avert a chronic attack. For example, historical data of stroke patients or cardiac arrest patients may indicate that a few days before a stroke or a cardiac arrest, the patient&#39;s normal activities slow down (e.g., number of steps walked, velocity of steps, activeness, social activities, etc.), their visits to the restroom are reduced, or their participation in social activities is reduced. If a given user is exhibiting similar symptoms, it may be highly likely that a few days later the given user may suffer from a stroke or a cardiac arrest. A timely diagnosis by a medical professional, for example, a doctor or nurse, along with a timely treatment may prevent or greatly reduce the changes of such a stroke or cardiac arrest. Moreover, certain unrecognized symptoms and problems may be identified. Operations end at Block  5076 . 
     Referring now to  FIG. 54 , in another embodiment, the controller  5037 ′ may also correlate the data trend with other data, for example, from one or more remote data sources  5019 ′. More particularly, the controller  5037 ′ may additionally correlate diet data associated with the user and use machine learning to predict the health change of the user also based upon the diet data. Diet data may include nutritional characteristics of food consumed by the user, for example. Diet data may include other and/or additional data about the diet of the user, for example, number of meals a day, times of meals, caloric intakes, etc. The diet data may be obtained by the controller  5037 ′ via a third party application, for example, a meal tracker application. The controller  5037 ′ may obtain the diet data via one more network, such as, for example, the Internet, and/or retrieved from a database associated with the third party. 
     The controller  5037 ′ may alternatively and/or additionally correlate medication data associated with the user, from the one or more remote data sources  5019 ′. The remote data source  5019 ′ may be a different data source or a same remote data source (e.g., server, application, etc.) than the remote data source with respect to the diet data. The controller  5037 ′ may thus use machine learning to predict the health change of the user also based upon the medication data. Medication data may include a type and amount of medications (e.g., prescription and non-prescription, supplemental, etc.). Similarly to the diet data, the medication data may be obtained by the controller  5037 ′ via a third party application, for example, a medication tracker application. The controller  5037 ′ may obtain the diet data via one more network, such as, for example, the Internet, and/or retrieved from a database associated with the third party, for example, a pharmacy and/or medical professional, such as a prescribing physician. 
     By including diet data and/or medication data in the correlation, a more accurate health change prediction may be achieved. For example, the correlation may determine what happens when a user eats a certain type of diet, say a diet with sugar or carbs. The data trends may suggest that a user&#39;s blood pressure and/or glucose levels or weight fluctuations are reduced when they eat a certain diet or when they follow an exercise schedule. The correlation may also determine that a user may feel more energetic, e.g., based upon a number of steps, when certain foods are consumed. The controller  5037 ′, based upon the correlation may also identify how changes in temperature (e.g., based upon thermostat set temperatures) affect the user. As will be appreciated by those skilled in the art, the correlation analysis may provide insights to doctors, nurses, or other medical professionals, for example, to prescribe certain lifestyle changes or changes in diets or food habits. For example, a relatively small change in diet or lifestyle may have a relatively large impact on older adult users. 
     Moreover, correlation analysis provides a relationship between two or more entities. While data trends may suggest that an individual, for example, walked more steps when their weight was reduced or vice-versa, it does not identify the cause, for example, is it that an individual lost weight because they walked more steps, or is that they were able to walk more steps because they lost weight and hence they felt more energetic. By analyzing the relevant data, causation analysis enables individuals, doctors/nurses, and/or other health professionals to understand a root cause of changes or fluctuation in a user&#39;s health (e.g., in real-time). For example, the causation analysis may determine whether the intake of more fats causing more negative health changes or complications for a given user, while the same diet may cause positive health changes for the another user. 
     Along with other analysis, both correlation and causation analysis serve as a basis for predicting how and when a user&#39;s health may change. For example, by knowing that a user consumed a certain diet or a certain food item that may be known to negatively change the user&#39;s health, staff, doctors, and nurses may be notified or alerted to an impending change in the user&#39;s health. Along with the data associated with a given user, data from similar (e.g., where the living area is a senior living facility, older adults) users may also be used to predict which set of other users will have what type of health changes and based upon the consumption of what type of food, for example. 
     Still further, correlation and causation data may enable dietitians, doctors/nurses, and/or other health professionals to better understand a user&#39;s health, for example, by helping them understand root causes of certain health problems for a given user, which may enable the prescription of lifestyle changes, diet changes, and/or other changes that may lead to positive health. In other words, as will be appreciated by those skilled in the art, the embodiments of the HA system  5020  described herein may be particularly advantageous for proposing diets or lifestyle changes based upon an individual user&#39;s biometrics, activities, and eating habits. 
     Referring now to  FIG. 55 , in another embodiment, the HA system  5020 ″ may also include a cloud server  5033 ″ remote from the HA hub device  5034 ″ in a cloud computing environment  5099 ″ and carrying the controller  5037 ″. In other words, the controller  5037 ″ may perform any of the functions described herein with respect to determining a data trend, correlating the data trend, and using machine learning to predict a health change, while not necessarily performing respective device functions, such as, for example, the functions of the HA operation devices  5031   a ″- 5031   n ″, the HA user interface device  5036 ″, and the HA hub device  5034 ″. Of course, in some embodiments, the controller  5037 ″ in the cloud computing environment  5099 ″ may perform at least some functions of the HA operation devices  5031   a ″- 5031   n ″, the HA user interface device  5036 ″, and/or the HA hub device  5034 ″. The functions described herein may also be split among multiple controllers, for example, one controller may be carried by the HA hub device  5034 ″, which another controller may be in the form of the cloud server  5033 ″. 
     A method aspect is directed to a method of predicting a health change of a user of an HA system  5020  that includes at least one HA operation device  5031   a - 5031   n , a biometric sensor  5059 , an HA hub device  5034  to provide communications for the at least one HA operation device. The method includes using at least one controller  5037  to cooperate with the biometric sensor  5059  to monitor a biometric characteristic of the user, and store historical operational data for the at least one HA operation device  5031   a - 5031   n  based upon the user. The method also includes using the at least one controller  5037  to determine a data trend of the at least one HA operation device  5031   a - 5031   n  based upon the stored historical operational data, and correlate the data trend with the biometric characteristic of the user. The method further includes using the at least one controller  5037  to use machine learning to predict the health change of the user based upon the correlated data trend and biometric characteristic of the user. 
     A computer readable medium aspect is directed to a non-transitory computer readable medium for predicting a health change of a user of an HA system  5020  that includes at least one HA operation device  5031   a - 5031   n , a biometric sensor  5059 , an HA hub device  5034  to provide communications for the at least one HA operation device  5031   a - 5031   n . The non-transitory computer readable medium includes computer executable instructions that when executed by at least one controller  5037  cause the at least one controller to perform operations. The operations include cooperating with the biometric sensor  5059  to monitor a biometric characteristic of the user, and storing historical operational data for the at least one HA operation device  5031   a - 5031   n  based upon the user. The operations also include determining a data trend of the at least one HA operation device  5031   a - 5031   n  based upon the stored historical operational data, and correlating the data trend with the biometric characteristic of the user. The operations further include using machine learning to predict the health change of the user based upon the correlated data trend and biometric characteristic of the user. 
     Further details of an HA system are described in U.S. patent application Ser. Nos. 15/628,328; 15/832,893; 16/176,224; 16/176,261; 15/834,887; 16/176,282; 16/176,315; 15/834,944; 15/990,207; 16/000,206; and 16/000,245 the entire contents of each of which are hereby incorporated by reference. 
     While several embodiments have been described herein, it should be appreciated by those skilled in the art that any element or elements from one or more embodiments may be used with any other element or elements from any other embodiment or embodiments. Many modifications and other embodiments of the invention will come to the mind of one skilled in the art having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is understood that the invention is not to be limited to the specific embodiments disclosed, and that modifications and embodiments are intended to be included within the scope of the appended claims.