Patent Description:
<CIT> relates to a haptic notification manager.

The invention is defined as set out by the appended claims. The present disclosure provides a framework for delivering notifications to a device or accessory based on the context. As used herein, an accessory may be used to refer to any electronic device that is coupled to a host device and capable of providing information in some form, such as audio, visual, haptic feedback, etc., to a user. Examples of accessories may include earbuds, smartwatches, headsets, other wearable electronics, etc. Accessories may further include other electronic devices that a user of the host device has access to, such as a laptop, hub, tablet, etc. For example, a host device may be wirelessly connected to one or more accessories such that the accessories are available to receive a notification transmitted from the host device. To determine which accessory to transmit a notification to, the host device may obtain data from a plurality of sensors. Thus, the host device may analyze the context of the accessories based on the sensor data. The host device may analyze user attention and analyses accessory state. The user attention may include, for example, whether the accessory is being actively used by the user, whether the accessory detects the user's gaze, the pose of the accessory, whether the accessory detects background chatter, and the location of the accessory. The accessory state includes at least one of do not disturb mode, silent mode or haptic mode, and may include, for example, whether the accessory is on and connected to the host device. The host device selects one or more of the connected accessories, based on the context, to transmit the notification to. The host device may select a different type of notification, based on the context, for each of the connected accessories. For example, the type of notification may include audio, visual, haptic, etc. Once transmitted, the notification may be dismissed by the user such that dismissing the notification on one accessory dismisses the notification across all devices and accessories.

One aspect of the disclosure provides for a host device comprising a short range communications interface adapted to transmit notifications to one or more accessories, a memory, and one or more processors in communication with the memory and the short range communications interface. The one or more processor are configured to identify which of the one or more accessories are available to receive a notification, analyze a context for transmitting the notification based on the one or more identified accessories, select at least one of the one or more identified accessories based on the context, and transmit the notification to the selected accessory.

The one or more processors may be further configured to request information from one or more sensors of each available device, the one or more sensors include at least one of (i) an audio input sensor, (ii) an image sensor, (iii) a touch sensor, or (iv) a proximity sensor, receive information from the one or more sensors, and determine, based on the received information, the context including at least one of user attention or an accessory state. User attention may include at least one of (i) active use of at least one of the one or more available accessories, (ii) gaze detection, (iii) pose detection, (iv) conversation detection, or (v) a location of the one or more available accessories. An accessory state includes at least one of (i) do not disturb mode, (ii) silent mode, or (iii) haptic mode.

The one or more processor may be further configured to determine a content type, determine, based on the content type, a content classification, and determine, based on the content type, the content classification, and the one or more identified accessories, a notification type to transmit to the selected accessory. The content type may include at least one of (i) a text message (ii) an e-mail, (iii) an application notification, (iv) a download, or (v) a news update. The content classification may include at least one of (i) urgent, (ii) sensitive, (iii) customary, or (iv) reminder. The notification type may include at least one of (i) a visual notification, (ii) an audio notification, or (iii) a haptic notification.

The one or more processors may be configured to receive content in the form of an electronic message or a data set, wherein the one or more processors are configured to determine the content type based on the received content. The one or more processors may be further configured to determine which of the one or more accessories are private accessories and public accessories. Private accessories may be accessible by a primary user of the device and public accessories are accessible to the primary user of the device and at least one other user.

The systems and methods described herein relate to a host device that may simultaneously couple to one or more accessories. As described above, an accessory may refer to any electronic device that is coupled to a host device and capable of providing information in some form, such as audio, visual, haptic feedback, etc., to a user. Examples of accessories may include earbuds, smartwatches, headsets, other wearable electronics, etc. Accessories may further include other electronic devices that a user of the host device has access to, such as a laptop, hub, tablet, etc. The accessories may be coupled to the host device via short-range communication, such as Bluetooth, Bluetooth low energy (BLE), etc. The host device and each of the connected accessories may include one or more sensors. Just as some examples, the sensors may include image sensor(s), a gaze detection sensor(s), audio detection input, touch sensor(s) and/or proximity sensor(s). According to some examples, Bluetooth signal strength analysis or ultra-wide band ("UWB") analysis may be used to detect when an accessory or electronic device is nearby. The sensor data may provide context relating to user attention. User attention may be inferred by, for example, active use of the accessory, detection of a user's gaze, pose data relating to an orientation of the accessory, detection of background chatter thereby alerting the host device that the user may be nearby, and the location of the accessory. Context data includes information pertaining to the accessory state. For example, the accessory state includes whether the accessory is turned on or off and whether the accessory is set to a certain mode, such as do not disturb, silent, haptic, etc. The host device may analyze the context data to determine which device or accessory to transmit a notification to.

<FIG> illustrates an example system <NUM> in which the features described herein may be implemented. It should not be considered limiting the scope of the disclosure or usefulness of the features described herein. In this example, system <NUM> may include a plurality of devices <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, server computing device <NUM>, and network <NUM>. The collection of devices may include, for example, smartphone <NUM>, hub <NUM>, smartwatch <NUM>, laptop <NUM>, and headset <NUM>. For purposes of ease, the collection of devices <NUM>, <NUM>, <NUM>, <NUM>, <NUM> or a single device will be referenced as device(s) <NUM>.

Each device <NUM> may be a personal computing device intended for use having all of the components normally used in connection with a personal computing device, as described herein, including a one or more processors (e.g., a central processing unit (CPU)), memory (e.g., RAM and internal hard drives) storing data and instructions, a display (e.g., a monitor having a screen, a touch-screen, a projector, a television, or other devices such as a smartwatch display that is operable to display information), and user input devices (e.g., a mouse, keyboard, touchscreen or microphone).

The devices <NUM> may each be a mobile computing device capable of wirelessly exchanging data with a server over a network such as the Internet, or a full-sized personal computing device. By way of example only, devices may include mobile phones, wireless-enabled PDAs, tablet PC, a netbook that is capable of obtaining information via the Internet or other networks, wearable computing devices (e.g., a smartwatch, headset, smartglasses, virtual reality player, other head-mounted display, etc.), wireless speakers, home assistants, gaming consoles, etc. In some examples, one or more of the devices <NUM> may be further wirelessly coupled to an accessory, such as earbuds, smartglasses, etc. Further, such accessory may be indirectly connected to the network <NUM> through the one or more devices <NUM>.

The devices <NUM> may be at various nodes of a network <NUM> and capable of directly and indirectly communicating with other nodes of network <NUM>. Although four (<NUM>) devices are depicted in <FIG>, it should be appreciated that a typical system <NUM> can include one or more devices, with each computing device being at a different node of network <NUM>. The network <NUM> and intervening nodes described herein can be interconnected using various protocols and systems, such that the network can be part of the Internet, World Wide Web, specific intranets, wide area networks, or local networks. The network <NUM> can utilize standard communications protocols, such as WiFi, that are proprietary to one or more companies. Although certain advantages are obtained when information is transmitted or received as noted above, other aspects of the subject matter described herein are not limited to any particular manner of transmission. In one example, system <NUM> may include one or more server computing devices having a plurality of computing devices, e.g., a load balanced server farm, that exchange information with different nodes of a network for the purpose of receiving, processing and transmitting the data to and from other computing devices. For instance, one or more server computing devices <NUM> may be a web server that is capable of communicating with the one or more devices <NUM> via the network <NUM>. In addition, server computing device <NUM> may use network <NUM> to transmit information to one of the other devices <NUM>. Server computing device <NUM> may include one or more processors, memory, instructions, and data. These components operate in the same or similar fashion as those described herein with respect to device <NUM>. The server computing devices <NUM> may receive sensor data from devices <NUM>. The server computing devices <NUM> may determine a context for transmitting a notification by analyzing the sensor data.

<FIG> illustrates an example system <NUM> in which the features described above and herein may be implemented. In this example, the system <NUM> may include host device <NUM> and accessories <NUM>, <NUM>, <NUM>, <NUM>, <NUM>. Host device <NUM> may contain one or more processors <NUM>, memory <NUM>, instructions <NUM>, data <NUM>, a wireless communication interface or antenna <NUM>, one or more sensors <NUM>, notification capabilities <NUM>, and device state <NUM>. The host device <NUM> may be able to communicate with accessories <NUM>, <NUM>, <NUM>, <NUM>, <NUM> via a short-range wireless communication interface.

The one or more processors <NUM> may be any conventional processors, such as commercially available microprocessors. Alternatively, the one or more processors may be a dedicated device such as an application specific integrated circuit (ASIC) or other hardware-based processor. Although <FIG> functionally illustrates the processor, memory, and other elements of host device <NUM> as being within the same block, it will be understood by those of ordinary skill in the art that the processor, computing device, or memory may actually include multiple processors, computing devices, or memories that may or may not be stored within the same physical housing. Similarly, the memory may be a hard drive or other storage media located in a housing different from that of host device <NUM>. Accordingly, references to a processor or computing device will be understood to include references to a collection of processors or computing devices or memories that may or may not operate in parallel.

Memory <NUM> may store information that is accessible by the processors, including instructions <NUM> that may be executed by the processors <NUM>, and data <NUM>. The memory <NUM> may be a type of memory operative to store information accessible by the processors <NUM>, including a non-transitory computer-readable medium, or other medium that stores data that may be read with the aid of an electronic device, such as a hard-drive, memory card, read-only memory ("ROM"), random access memory ("RAM"), optical disks, as well as other write-capable and read-only memories. The subject matter disclosed herein may include different combinations of the foregoing, whereby different portions of the instructions <NUM> and data <NUM> are stored on different types of media.

Data <NUM> may be retrieved, stored or modified by processors <NUM> in accordance with the instructions <NUM>. For instance, although the present disclosure is not limited by a particular data structure, the data <NUM> may be stored in computer registers, in a relational database as a table having a plurality of different fields and records, XML documents, or flat files. The data <NUM> may also be formatted in a computer-readable format such as, but not limited to, binary values, ASCII or Unicode. By further way of example only, the data <NUM> may be stored as bitmaps comprised of pixels that are stored in compressed or uncompressed, or various image formats (e.g., JPEG), vector-based formats (e.g., SVG) or computer instructions for drawing graphics. Moreover, the data <NUM> may comprise information sufficient to identify the relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other memories (including other network locations) or information that is used by a function to calculate the relevant data.

The instructions <NUM> can be any set of instructions to be executed directly, such as machine code, or indirectly, such as scripts, by the processor <NUM>. In that regard, the terms "instructions," "application," "steps," and "programs" can be used interchangeably herein. The instructions can be stored in object code format for direct processing by the processor, or in any other computing device language including scripts or collections of independent source code modules that are interpreted on demand or compiled in advance.

The host device <NUM> may further include a wireless communication interface <NUM>, such as an antenna, transceiver, and any other devices used for wireless communication. The antenna may be, for example, a short-range wireless network antenna. The host device <NUM> may be able to be coupled with accessories <NUM>, <NUM>, <NUM>, <NUM>, <NUM> via wireless connections <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, respectively. For instance, the antenna may be used to transmit and receive Bluetooth signals. There may be a maximum distance between host device <NUM> and accessories <NUM>, <NUM>, <NUM>, <NUM>, <NUM> that would allow host device <NUM> and accessories <NUM>, <NUM>, <NUM>, <NUM>, <NUM> to be within range of each other.

The host device <NUM> may include one or more sensors <NUM>. The sensors may include, for example, image sensor(s), gaze detection sensor(s), audio detection sensor(s), touch sensor(s), and proximity sensor(s). The data obtained by the sensors may be analyzed to determine a context. The context includes user attention and/or accessory state. User attention may include, for example, detection of a user's proximity and/or a user's gaze, detection of background noise, positioning of the host device <NUM> in comparison to the connected accessories, etc..

The one or more image sensors may convert optical signals into electrical signals to detect, or capture, features and/or objects around host device <NUM>. The one or more image sensors may be, for example, a charge coupled device ("CCD") sensor or a complementary metal oxide semiconductor ("CMOS") sensor. The one or more processors <NUM> may process the features and/or objects detected by the one or more image sensors to identify at least one detected feature and/or object. The one or more image sensors may be used to detect a user's gaze such that the image sensors also function as a gaze detection sensor. According to some examples, the image sensors may be used to determine the pose of the device.

The audio detection sensor may include, for example, one or more microphones configured to receive audio input signals. The audio input may include background noise, such as music, people talking, etc. The user can enable or disable receiving background noise as audio input, and the device may only receive background noise as audio input if the user has enabled this feature. According to some examples, a user may adjust settings on the device to restrict sharing to particular geographical areas, times, or conditions, such as when the device is within Bluetooth range of a predetermined number of other user devices. Moreover, the user may set the one or more microphones to automatically disable in particular locations. According to some examples, the user may set the device to automatically disable receiving background noise as audio input in particular locations, such as familiar locations. The user may further configure the device to receive audio input without further active input from the user. In some examples, the microphone used for audio detection input may also be used for user input.

The proximity sensor may detect the proximity of the accessory to other devices, such as based on signal strength or other communication parameters.

The sensor data may be analyzed by the host device <NUM> or another device on the network to determine user attention. Analyzing the sensor data may include, for example, determining whether a user is likely to see, hear, or feel a notification. Additionally or alternatively, analyzing the sensor data may include selecting an accessory to receive a notification. For example, based on the user attention, the host device <NUM> may transmit a notification to a connected accessory. According to some examples, based on the user attention the host device <NUM> may output a notification.

Host device <NUM> may include one or more notification capabilities or settings <NUM>. The notifications may be, for example, audio, haptic, visual, or a combination of audio, haptic, or visual. Audio notifications output by host device <NUM> may include a noise or a sound that can be heard by the user. Thus, host device <NUM> may further include one or more outputs, such as speakers. Host device <NUM> may further include controls to determine a volume level of the audio notification.

The one or more devices states <NUM> include at least one of do not disturb, silent, or haptic mode, and may include off mode, etc. For example, when host device <NUM> is off, host device <NUM> may be unable to transmit or receive notifications. In some examples, when host device <NUM> is set to do not disturb, notifications received by host device <NUM> may not result in a notification provided to the user. When in silent mode, host device <NUM> may receive content and notifications but may only provide a visual notification. When in haptic mode, host device <NUM> may receive content and notifications but may only provide a haptic notification. Host device <NUM> may have a default state which may, for example, allow notifications of any type, including audio, haptic, or visual.

Accessories <NUM>, <NUM>, <NUM>, <NUM>, <NUM> may each include one or more processors <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, memory <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, wireless communication interface <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, sensors <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, notification capabilities or settings <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and device state <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, respectively, that are substantially similar to those described herein with respect to host device <NUM>.

Host device <NUM>, or another device on the network, may analyze the sensor data of each of the accessories to determine user attention. The accessories may be ranked based on user attention. A high ranking accessory may be an accessory that the user is likely to hear, see, or feel a notification whereas a low ranking accessory may be, for example, an accessory that the user is not likely to hear, see, or feel a notification. For example, a high ranking accessory may be an accessory that the user is actively using. A low ranking accessory may be an accessory that does not detect the user's presence or gaze.

Analyzing the sensor data may include, for example, determining which, if any, accessory is actively being used. Analyzing the sensor data may also include determining a proximity of the accessory to the host device <NUM>. For example, an accessory that is closer in proximity to the host device <NUM> may rank higher in user attention than an accessory that is farther away. According to some examples, analyzing the sensor data may include detecting the presence of the user and/or detecting a user's gaze. An accessory that detects the presence of a user and/or the user's gaze using the image sensors may rank higher in user attention than an accessory that does not detect the presence of the user and/or the user's gaze.

Host device, or another device on the network, analyses the accessory state. Analyzing the accessory state may include determining which accessories are off or which are set to do not disturb, silent, haptic, morning, evening, etc. For example, the host device, a network computing device, or another device on the network may send a request to each accessory for the status of the accessory. The accessory may respond to the request with an indication of the accessory state. Additionally or alternatively, accessories that are not reachable by a status request may be determined to be offline and/or off. Accessories that are off may be excluded from the list of available accessories to receive a notification. According to some examples, the accessory state may be set by a user. In some examples, an application on the host device may be used to specific notification settings for the connected accessories. In another example, a cloud managed state system may be used to determine the accessory state. For example, each accessory may access a cloud managed account that includes the accessory state for each accessory.

The accessory state is used as a factor for determining which accessory to transmit a notification to. The accessory state, such as do not disturb, morning, evening, etc., may result in each accessory responding differently and, therefore, may affect the accessory's ranking for transmitting a notification to. For example, accessories that are set to do not disturb may rank lowest for transmitting a notification to. According to some examples, the host device may determine an accessory set to silent may only receive a visual notification. In some examples, when the accessory is set to haptic mode, the host device may determine the accessory can receive a visual and haptic notification.

Accessory state and optionally user attention determine a context for host device to transmit a notification to one of the accessories. The context may, for example, determine which accessory a user will most likely see, hear, or feel a notification. Based on the accessory state and optionally the user attention, host device <NUM>, or another device on the network determine which accessory to receive a notification.

<FIG> illustrates an example system <NUM> of a host device and a plurality of accessories connected via a short-range communications interface. The host device may categorically store paired accessories. Once the host device identifies which accessories are connected, the host device may categorize the connected accessories based on whether the accessories are private or public. For example, a private accessory may be an accessory that is intended for use by the user of the host device only, whereas a public accessory may be an accessory that is intended for use by the user of the host device as well as other individuals. In further examples, the accessories may be further categorized based on user preference, the type of accessory, etc..

The host device may be, for example, a smartphone <NUM>, or any of a variety of other host devices, such as laptops, gaming systems, tablets, etc. Smartphone <NUM> may have a short-range wireless communications interface. The short-range communications interface may include, for example, a wireless network antenna, transceiver, etc. The short-range communications interface may allow smartphone <NUM> to wirelessly connect to the accessories. The accessories may include a home assistant hub <NUM>, a smartwatch <NUM>, a laptop <NUM>, an augmented reality and/or virtual reality headset <NUM>, and a pair of earbuds <NUM>, or any of a variety of other accessories, such as fitness trackers, wireless speakers, glasses, head-mountable displays, tablets, game consoles, motorcycle helmets, etc. Each accessory may include a short-range communications interface that allows the accessory to be coupled to the host device. The wireless coupling may be established using any of a variety of techniques, such as Bluetooth, Bluetooth low energy (BLE), etc. The accessories may be categorized as private or public. According to some examples, the accessories may be categorized based on user input.

The host device may be wirelessly connected to one or more accessories simultaneously. For example, smartphone <NUM> may be connected to earbuds <NUM> via wireless connection <NUM>, to hub <NUM> via wireless connection <NUM>, to smartwatch <NUM> via wireless connection <NUM>, to laptop <NUM> via wireless connection <NUM>, and to headset <NUM> via wireless connection <NUM>. According to some examples, smartphone <NUM> may be wirelessly connected to hub <NUM>, smartwatch <NUM>, laptop <NUM>, headset <NUM>, and earbuds <NUM> simultaneously when such accessories are within the range of the wireless network antenna. In some examples, smartphone <NUM> may only be connected to earbuds <NUM> and smartwatch <NUM>, for example, due to a range of its wireless network antenna. Thus, smartphone <NUM> may be wirelessly coupled to any number of accessories within range of its wireless network antenna.

Smartphone <NUM> may store a categorical list <NUM> of the connected accessories. Just as one example, the list <NUM> may identify devices as private devices <NUM> and public devices <NUM>. A private accessory may be an accessory that typically outputs content only to a primary user. The primary user may be determined based on a user profile that was set up on the host device during the out of box experience. According to some examples, earbuds or a smartwatch may be a private accessory. For example, earbuds or smartwatch may be accessories that are only used by the primary user. Thus, content or notifications transmitted to the earbuds or smartwatch may only be received by the primary user. A public, or shared, accessory may be an accessory that typically outputs content to anyone within audio, visual, or other output range of the accessory. By way of example, wireless speakers or displays, short-range wireless stereos, etc. may be shared accessories.

The list <NUM> may, in some examples, include identifiers indicating how each accessory is categorized. The identifier may be, for example, a color, an icon, text, placement of the accessory name in a particular section, heading, sub-list, etc. According to some examples, the list <NUM> may be determined by user input. For example, the user may provide a designation of each device or accessory after that device or accessory is connected to the host device. The list <NUM> may be modified by a user such that the user may change the designation of devices, add devices, or remove devices at any time.

As shown in system <NUM>, smartphone <NUM> may categorically store the connected earbuds <NUM>, hub <NUM>, smartwatch <NUM>, headset <NUM>, and laptop <NUM>. For example, smartphone <NUM> may categorize earbuds <NUM>, smartwatch <NUM>, and headset <NUM> as private accessories <NUM> and may categorize hub <NUM> and laptop <NUM> as public accessories <NUM>.

<FIG> illustrates a system in which the context of the host device and one or more accessories may be analyzed. The context may include user attention and accessory state. According to some examples, the host device, server computing device, or another device connected to the network may analyze the context data. Analyzing context data may include, for example, analyzing sensor data collected by each of the accessories.

User attention may be inferred by, for example, active user of the host device and/or accessory, detection of a user's gaze by the host device and/or accessory, pose data pertaining to the device or accessory, whether the host device or accessory picks up background chatter, the proximity of the accessory to the host device, etc. Data pertaining to user attention may be determined based on the data from the one or more sensors of the host device or accessory. The data may be transmitted or shared over the network to the host device, a server computing device, or another device on the network to be analyzed.

An accessory state may be inferred by, for example, whether the accessory is turned on or off, whether the accessory is set to do not disturb, silent, haptic mode, etc. Data pertaining to the accessory state may be transmitted to the host device, the server computing device, or another device on the network to be analyzed.

The data may be collected from the host device and accessories and shared via the network. The host device may analyze the context data to determine a context for delivering a notification to the host device and/or at least one accessory. According to some examples, the data may be analyzed by the server computing device or another device on the network.

The host device may rank the available accessories based on each accessory's respective context data. The rank may, in some examples, determine the accessory to which the host device will transmit a notification. For example, the sensor data and accessory state may be analyzed to determine a rank of each accessory. According to some examples, an accessory that is closer in proximity to the host device may be ranked higher than an accessory that is located further away. An accessory that is actively being used may be ranked higher than an accessory that is off or set to do not disturb. An accessory that detects the presence of the user and/or the user's gaze may be ranked higher than an accessory that does not detect the presence of the user and/or the user's gaze.

According to some examples, the context data may also be analyzed to rank each accessory based on the determined type of notification to be transmitted. For example, an accessory that detects the presence of the user and/or the user's gaze may be ranked higher for receiving a visual notification as the user may easily see the notification if sent to that accessory. An accessory that is on the user or being worn by the user may be ranked higher for receiving a haptic notification than an accessory that is located at a distance from the user as the user may easily feel the notification.

<FIG> illustrates a system in which the host device receives content and determines a content type. The host device may receive content that is sent from another device, sent from an application, sent via server computing device over the network, etc. For example, the content may be received in the form of an electronic message or a data set received by the host device. The one or more processors of the host device may determine, based on the received content, a content type. The content type may include, for example, text message, e-mail, calendar notifications and/or reminders, news updates, application updates, a phone call, etc. According to some examples, determining the content type may include determining an application that has received content. For example, text messages may be received in a message application, e-mails may be received in an e-mail application, calendar content may be from a calendar application, news content may be received in a news application, application content may be from an application or play store, etc. Once content is received, an indication may appear on the application. According to some examples, the server computing device or another device connected to the network may determine the content type.

System <NUM> may include a host device, such as smartphone <NUM>, connected to network <NUM>. Smartphone <NUM> may include a plurality of applications, such as messages <NUM>, e-mail <NUM>, calendar <NUM>, news <NUM>, application store <NUM>, phone <NUM>, games, social media applications, etc. The applications may receive content intended for output to the user. For example, the messages application <NUM> may receive text, picture, or video messages, the e-mail application <NUM> may receive e-mails with a variety of content intended for the user, the calendar application <NUM> may receive invitations or reminders, the news application <NUM> may receive real-time content, the application store <NUM> may receive updates to the applications on smartphone <NUM>, and phone <NUM> may receive video and/or voice calls.

Smartphone <NUM> may receive content, such as a text message, sent via network <NUM>. The one or more processors <NUM> of smartphone <NUM> may determine that the content is a text message and, therefore, should appear in the messages application <NUM>. The smartphone <NUM> may provide an indication <NUM> that the smartphone <NUM> has received content. The indication <NUM> may be provided on the messages application <NUM> thereby indicating that the content is a text message.

<FIG> illustrates a system in which the one or more processors of the host device determines a content classification of the received content. The content classification may include designating the content as urgent, sensitive, reminder, customary, etc. According to some examples, the server computing device or another device connected to the network may determine the content classification of the content received by the host device. Additionally or alternatively, the content type may be set by the user for each application. For example, content received by the text message application may be indicated as urgent, which updates from a news application may be considered customary.

Urgent content may be content sent from specific people or applications that may be automatically selected or designated by the user of the host device. For example, the host device may determine a list of top ten (<NUM>) people from the contact list that the user communicates. Any content sent from those top ten people may be designated as urgent. According to some examples, the list may be five (<NUM>) designated people, twelve (<NUM>) designated people, twenty (<NUM>) designated people, twenty-five (<NUM>) designated people, etc. Therefore, a list of ten people is merely one example and is not meant to be limiting. According to some examples, urgent content may be determined based on the semantic analysis of the content and the source of the content. For example, a message from the user's manager reading "Can you talk now about project X?" may be classified as urgent.

Sensitive content may include content that requires privacy, such that it should only be provided to a user of the host device as opposed to other individuals in the company of the user. According to some examples, sensitive content may be any content received by the host device by certain applications or profiles. For example, content received by the host device for a work profile may automatically or selectively be set to sensitive. According to some examples, sensitive content may be, for example, more or less sensitive based on the context. For example, content from particular users or contacts may be considered sensitive when in a public location but may not be considered sensitive when at the user's home. In one example, a message from a primary contact about a grocery list may be public content when in the user's home but may be classified as private when in a public location, such as work.

Reminders may be content received by the host device based on a certain date or time. For example, the calendar application may provide reminder content regarding a scheduled appointment, meeting, gathering, etc. According to some examples, the e-mail application may provide reminder content regarding a follow-up e-mail.

Customary content may be any content received by the host device that is not designated as urgent, sensitive, reminder, etc..

The one or more processors <NUM> may determine the content classification based on the received content. For example, the received content may be a text message, as shown by indication 526a on the messages application <NUM>. The one or more processors <NUM> may determine whether that text message is urgent, sensitive, customary, or reminder content. For example , the content may be classified as urgent. The indication 526a may include a symbol showing the content as urgent. For example, indication 526a may include an exclamation mark (!) to show that the content is urgent.

<FIG> illustrates a system in which the host device, using one or more processors, may determine a notification type. The notification type may be determined based on one or more of the analyzed context data, the determined content type, and the determined content classification. The notification type may be, for example, any one of or a combination of a haptic, audio, or visual notification. The host device, using the one or more processors, determines which accessory to transmit to the notification to.

According to some examples, the host device, the server computing device, or another device on the network may include an algorithm that determines which accessory to receive the notification. The algorithm may be based on the context data collected from the plurality of sensors, the content type, the content classification, and/or the notification type. Additionally or alternatively, a mesh network that hosts the host device and the one or accessories may include the algorithm to determine which accessory to receive the notification. In some examples, the mesh network may be a local mesh network.

For example, a notification for urgent content may be transmitted to an accessory with one or more notification type abilities. Urgent content may cause the host device to transmit a haptic, audio, and visual notification to alert a user of the content. Additionally or alternatively, the host device may transmit the notification regarding the urgent content to an accessory that is currently being used and/or looked at. For example, if an accessory detects the user's gaze, the host device may transmit the notification to that accessory as it is likely the user will see the notification.

According to some examples, a notification for sensitive content may be transmitted to a private accessory in an attempt to ensure that only the user of the host device receives the sensitive content.

A notification for reminder content may be transmitted to an accessory with audio and visual notification capabilities. For example, the audio notification may draw the attention of the user and the visual notification may provide the reminder.

A notification for a customary reminder may be transmitted to any accessory, regardless of whether the accessory has been classified as private or public. The notification type for customary content may be based upon the available accessories. In some examples, the notification type may be a combination of notification types.

While not described, any number of combinations of notification types may be transmitted from the host device to an accessory based on one or more of the analyzed context data, the determined content type, and the determined content classification. Thus, the examples provided herein are not intended to be limiting.

The one or more processors <NUM> of smartphone <NUM> may determine a notification type to transmit. For example, the one or more processors <NUM> may determine whether to transmit a haptic, audio, visual, or combination notification. The notification type may be determined based on one or more of the context data, the content type, the content classification, and the available accessories.

As shown by indication 526a, the message application <NUM> has received urgent content. The one or more processors <NUM>, based on the context data, content type, available accessories, and/or content classification, may determine a notification type. For example, the processors <NUM> may determine to transmit a haptic and visual notification based on the content being a text message, the text message being urgent, and the notification capabilities of the available accessories.

Once the notification type has been determined, smartphone <NUM> may determine which accessory to transmit the notification to. Smartphone <NUM> may transmit the notification to at least one of smartwatch <NUM>, earbuds <NUM>, hub <NUM>, headset <NUM>, and laptop <NUM> as those accessories are wirelessly connected to smartphone <NUM>. As the content was determined to be urgent content, smartphone <NUM> may transmit the notification to a private accessory, such as smartwatch <NUM> or earbuds <NUM>. Further, as the content was received by the messages application <NUM>, the smartphone <NUM> may determine to transmit at least a visual notification. The visual notification may include a preview of the content. As shown, smartphone <NUM> may transmit the notification <NUM> to smartwatch <NUM>. The notification <NUM> may include a haptic notification 628a and a visual notification 628b.

<FIG> illustrates an example where a host device determines which accessory to send a notification to based on one or more of the analyzed context data, the determined content type, the determined content classification, the determined notification type, and the available accessories.

As shown in <FIG>, smartphone <NUM> may not be being actively used by user <NUM>. For example, smartphone <NUM> may be in a bag or backpack at the feet of or behind the user <NUM>. Smartphone <NUM> may receive content <NUM>.

Smartphone <NUM> may identify which accessories are available to receive a notification. Thus, smartphone <NUM> may identify smartwatch <NUM>, earbuds <NUM>, and laptop <NUM> as available accessories as they may be wirelessly connected to smartphone <NUM>. Smartphone <NUM> may categorize the accessories as public or private. A public accessory may be an accessory in which notifications and content may be accessible to individuals in addition to the smartphone user. A private accessory may be an accessory in which notifications and content may be accessible to only the smartphone user. For example, the smartwatch <NUM> and earbuds <NUM> may be categorized as private accessories as the content and/or notification may only be accessible to user <NUM> whereas laptop <NUM> may be categorized as a public accessory as individuals other than user <NUM> may be able access, see, or hear the content and/or notification.

Smartphone <NUM> may analyze a context for transmitting the notification based on the available accessories. Context data may be analyzed by smartphone <NUM>, a server computing device, or another device on the network based on data from one or more sensors of each of the accessories. The context may provide a framework for determining a relevant, available, or accessible accessory to transmit a notification to. The context includes accessory state and may include user attention.

User attention may include whether the accessory is being actively used, whether the accessory detects the gaze of a user, the pose of the accessory, whether the accessory detects background chatter, the location. The accessory state includes whether the accessory is set to at least one of do not disturb, silent, or haptic mode, and may include whether the accessory is turned on or off.

For example, the smartwatch <NUM> may detect that it is being worn by user <NUM>. However, the pose of smartwatch <NUM> may be such that the display is not facing the user <NUM>. Earbuds <NUM> may detect that they are being worn by user <NUM>. Laptop <NUM> may have an image sensor with a field of view <NUM>. The image sensor may detect the presence of user <NUM> within field of view <NUM>. Additionally or alternatively, the image sensor may detect the gaze of user <NUM> when user <NUM> is looking at laptop <NUM>.

Smartphone <NUM> may select at least one or more of smartwatch <NUM>, earbuds <NUM>, and laptop <NUM> based on the context. For example, smartphone <NUM> may rank smartwatch <NUM>, earbuds <NUM>, and laptop <NUM> based on the context data. Laptop <NUM> may be ranked highest as user <NUM> is actively engaging with and, therefore, gazing at the display of laptop <NUM>. Earbuds <NUM> and smartwatch <NUM> may be ranked below laptop <NUM>. For example, smartwatch <NUM> may be ranked lower than laptop <NUM> as the display of smartwatch <NUM> is facing away from user <NUM>. Thus, user <NUM> may not be actively engaging with smartwatch <NUM>, further leading to a lower rank. Earbuds <NUM> may be ranked lower than laptop <NUM> as user <NUM> may not be actively engaging with earbuds <NUM>.

Smartphone <NUM> may select which accessory to transmit a notification to based on the content type. For example, a notification for a phone call may be transmitted to earbuds <NUM> even though earbuds <NUM> may be ranked lower than laptop <NUM>. Earbuds <NUM> may be best suited for user <NUM> to interact with the notification, such as by answering the call. A notification for a text message may be transmitted to smartwatch <NUM> even though smartwatch <NUM> may be ranked lower than laptop <NUM>. A notification for an email may be transmitted from smartphone <NUM> to laptop <NUM>. According to some examples, laptop <NUM> may be best suited for user <NUM> to interact with the notification, such as by replying to the email.

In some examples, smartphone <NUM> may select which accessory to transmit the notification to based on the content classification. For example, the content may be classified as urgent, sensitive, customary, reminder, etc. Smartphone <NUM> may transmit a notification for urgent content to an accessory that user <NUM> is currently engaging with to ensure that user <NUM> receives the notification. For example, a notification for urgent content may be transmitted to laptop <NUM> based on the image sensor detecting the user's gaze. Thus, user <NUM> may be likely to see the notification on laptop <NUM>. Smartphone <NUM> may transmit a notification for sensitive content to a private accessory that is accessible or viewed only by user <NUM>. For example, the notification may be transmitted to earbuds <NUM> or smartwatch <NUM>. Smartphone <NUM> may transmit a notification for customary or reminder content to any accessory, regardless of whether the accessory is categorized as private or public.

Smartphone <NUM> may select a notification type to transmit to the accessory. For example, earbuds <NUM> may be capable of receiving audio and/or haptic notifications, laptop <NUM> may be capable of receiving audio and/or visual notifications, and smartwatch <NUM> may be capable of receiving audio, visual, and/or haptic notifications.

After analyzing the context data, content type, content classification, and/or notification type, smartphone <NUM> may transmit a notification <NUM> for content <NUM> to laptop <NUM>. As laptop <NUM> may detect the gaze of the user, the notification <NUM> may be a visual notification 728b. The visual notification <NUM> may be a text box or a pop-up on the laptop <NUM> display.

The laptop <NUM> may receive an input from the user in response to the notification <NUM>, 728a. For example, laptop <NUM> may receive a command to ignore, dismiss, mute, respond, etc. The input received by the laptop <NUM> with regards to notification <NUM>, 728a may be executed on the laptop <NUM>, as well as the host device <NUM>, smartwatch <NUM>, earbuds <NUM>, etc. even if the notification <NUM>, 728a was not sent to those devices. For example, laptop <NUM> may receive an input to dismiss notification <NUM>, 728a, the content and/or notification on smartphone <NUM> may also be dismissed. Additionally or alternatively, laptop <NUM> may receive an input to dismiss the notification <NUM>, 728a, and, therefore, smartwatch <NUM> may dismiss the content and/or notification, even though a notification was never presented to user <NUM> on smartwatch <NUM>. Thus, an input received on the accessory receiving the notification will be executed across all devices and accessories that are coupled.

<FIG> illustrates another example where a host device determines which accessory to transmit a notification to. For example, there may be a host device, such as smartphone <NUM>, and a host device user <NUM>. Host device user <NUM> may be wearing smartwatch <NUM> and earbuds <NUM>. Host device user <NUM> may be looking at or interacting with hub <NUM>. Smartphone <NUM> may be wirelessly connected to smartwatch <NUM>, earbuds <NUM>, and hub <NUM>. Smartphone <NUM>, smartwatch <NUM>, earbuds <NUM>, and hub <NUM> may also be connected to a network. Thus, smartphone <NUM>, smartwatch <NUM>, earbuds <NUM>, and hub <NUM> may transmit and receive content from each other via the wireless communications interface and/or via the network.

An additional user <NUM> may be nearby host device user <NUM>. The additional user <NUM> may have their own host device, such as smartphone <NUM>, smartwatch <NUM>, and earbuds <NUM>. Smartphone <NUM> may be wirelessly connected to smartwatch <NUM> and earbuds <NUM>. Smartphone <NUM>, smartwatch <NUM>, and earbuds <NUM> may not be, according to some examples, wirelessly connected to hub <NUM>. The additional user <NUM> may be able to view or interact with hub <NUM> even when the additional user's devices are not connected with hub <NUM>.

Smartphone <NUM> may be in a bag or backpack located at a distance from user <NUM>. Thus, user <NUM> may not be currently interacting with, using, or able to view content on smartphone <NUM>. Smartphone <NUM> may, however, receive content <NUM>. Smartphone <NUM> may determine to transmit a notification to an accessory to notify user <NUM> of the received content <NUM>.

Smartphone <NUM> may identify which accessories are available to receive a notification. For example, smartphone <NUM> may identify that it is wirelessly connected to earbuds <NUM>, smartwatch <NUM>, and hub <NUM> and, therefore earbuds <NUM>, smartwatch <NUM>, and hub <NUM> are available accessories. Smartphone <NUM> may identify smartwatch <NUM>, earbuds <NUM>, and smartphone <NUM> as accessories capable of coupling but are not wirelessly connected to smartphone <NUM> and, therefore, are not available accessories.

Smartphone <NUM> may analyze the context for transmitting the notification. The context includes accessory state and may include user attention.

User attention and accessory state may be based on, for example, data collected by sensors on each of the accessories. For example, the accessories may have an audio input sensor, an image sensor, a proximity sensor, etc. The sensors may, for example, detect user attention attributes, such as whether the accessory is actively being used, detect the gaze of users nearby, detect the pose of the accessory, detect background noise, compare the location of the accessory to the location of the host device, etc. According to some examples, each accessory, hub <NUM>, earbuds <NUM>, and smartwatch <NUM> may be set to a certain state, or mode, such as do not disturb, silent, haptic, etc. The sensor data and current state of the accessory may be analyzed by smartphone <NUM> or a server computer to determine the user attention and/or accessory state with respect to hub <NUM>, earbuds <NUM>, and smartwatch <NUM>.

Hub <NUM> may have one or more image sensors that may detect objects within the field of view <NUM> of the image sensor. As shown, both host device user <NUM> and additional user <NUM> may be within the field of view <NUM> of the image sensor of hub <NUM>. Thus, hub <NUM> may detect the presence of users <NUM>, <NUM>. Hub <NUM> may also include one or more audio input sensors, such as a microphone. The microphone may detect the lull of chatter in the background, thereby indicating that users <NUM>, <NUM> are present. Hub <NUM> may additionally or alternatively include a proximity sensor to determine its proximity to smartphone <NUM>.

Earbuds <NUM> may include one or more audio input sensors, such as a microphone, to pick up background noise. The background noise may include, for example, the lull of chatter, music playing, etc. The background noise may indicate that someone, other than user <NUM>, is nearby.

Smartwatch <NUM> may include a sensor to determine whether smartwatch <NUM> is being worn by user <NUM>. Smartwatch <NUM> may also include a proximity sensor. The proximity sensor may determine the proximity of the smartwatch <NUM> to smartphone <NUM>.

Smartphone <NUM> may determine a type of content. For example, smartphone <NUM> may determine whether it has received a text, e-mail, news update, application update, phone call, etc. Once the content type is determined, smartphone <NUM> may determine a content classification, such as whether the content is urgent, sensitive, customary, a reminder, etc. Based on the type of content, the content classification, and/or the notification capabilities of the available accessories, smartphone <NUM> may determine a notification type. For example, hub <NUM> may be capable of audio and/or video notification, smartwatch <NUM> may be capable of audio, visual, and/or haptic notifications, and earbuds <NUM> may be capable of audio and/or haptic notifications.

According to some examples, content <NUM> may be an incoming phone call intended for user <NUM>. Smartphone <NUM> may analyze the context for transmitting a notification regarding a phone call. For example, sensor data from hub <NUM> may indicate that there are at least two users <NUM>, <NUM> present based on the field of view <NUM> of the image sensors. Sensor data from smartwatch <NUM> may indicate that user <NUM> is not actively engaging with smartwatch <NUM> based on its pose data. Additionally or alternatively, sensor data from smartwatch <NUM> may indicate that there is at least one additional user <NUM> present due to the background noise picked up by the microphones. Earbuds <NUM> may recognize that they are currently being worn by user <NUM>. As content <NUM> is a phone call, earbuds <NUM> may allow user <NUM> to easily interact with a notification. For example, earbuds <NUM> may be used by user <NUM> to have a conversation after answering the call.

Smartphone <NUM> may transmit a notification <NUM> to earbuds <NUM>. The notification <NUM> may be an audio notification 828c that can be heard by user <NUM> wearing earbuds <NUM>. The notification <NUM>, 828c may alert user <NUM> of the phone call received by smartphone <NUM>.

Earbuds <NUM> may receive an input by user <NUM> to respond to the notification. For example, the user may provide a touch input, such as a swipe, to answer the phone call or dismiss the phone call. The input provided by the user may be applied to any other devices that may have received the content <NUM> and/or notification. Thus, user <NUM> answering the call on earbuds <NUM> would remove the content <NUM> and/or notification from smartphone <NUM>. Additionally or alternatively, user <NUM> answering or dismissing the call on earbuds <NUM> would remove the content <NUM> and/or notification from smartwatch <NUM> and hub <NUM>.

<FIG> illustrates an example method for determining an accessory to transmit a notification based on the context. In block <NUM>, a host device identifies one or more accessories are available to receive a notification. The available accessories may be accessories that are wirelessly coupled to the host device via a short-range communications interface.

In block <NUM>, the host device analyses a context for transmitting the notification based on the one or more identified accessories. For example, analyzing the context may include analyzing data from one or more sensors including an audio input sensor, image sensor, proximity sensor, etc. The context for transmitting the notification includes an accessory state and may include user attention. User attention may include, for example, at least one of active use of the accessories, gaze detection, pose detection, conversation detection, the location of the device, etc. User attention may be determined based on the data collected from the sensors. The accessory state includes at least one of do not disturb, silent, or haptic.

In block <NUM>, the host device may determine a content type. The content type may include, for example, at least one of a text message, an e-mail, an application notification, a download, a news update, etc..

In block <NUM>, the host device may determine a content classification. The content classification may be based on the content type or semantic analysis of the content. According to some examples, the content classification may be urgent, sensitive, customary, reminder, etc..

In block <NUM>, the host device may determine a type of notification to transmit to the selected accessory. The type of notification may be based on the type of content, the classification of content, and/or the identified accessories. The notification type may be, for example, at least one of a visual, audio, or haptic notification.

In block <NUM>, the host device selects at least one of the identified accessories based on the context. Additionally the host device may select one of the identified accessories based on the content type, content classification, and/or notification type.

In block <NUM>, the host device transmits the notification to the selected accessory.

While the method was described with respect to the host device, a server computing device may perform some or all of the method steps.

Claim 1:
A method performed by a host device (<NUM>; <NUM>), the method comprising:
identifying (<NUM>), by one or more processors (<NUM>) of the host device (<NUM>; <NUM>), which of one or more accessories (<NUM> - <NUM>; <NUM> - <NUM>) are available to receive a notification (<NUM>; <NUM>);
analyzing (<NUM>), by the one or more processors (<NUM>) based on the one or more identified accessories (<NUM> - <NUM>; <NUM> - <NUM>), a context for transmitting the notification (<NUM>; <NUM>; <NUM>), wherein the context includes an accessory state (<NUM>-<NUM>) of one or more accessories, wherein the accessory state (<NUM> - <NUM>) includes at least one of do not disturb mode, silent mode, or haptic mode;
selecting (<NUM>), by the one or more processors (<NUM>) based on the context, at least one of the one or more identified accessories (<NUM> - <NUM>; <NUM> - <NUM>); and
transmitting (<NUM>), by the one or more processors (<NUM>), the notification (<NUM>; <NUM>; <NUM>) to the selected at least one of the one or more identified accessories (<NUM> - <NUM>; <NUM> - <NUM>).