Source: https://patents.google.com/patent/US9853931B2/en
Timestamp: 2019-08-22 12:17:23
Document Index: 404104548

Matched Legal Cases: ['application No. 10', 'Application No. 2014206208', 'Application No. 2014206208', 'Application No. 10', 'Application No. 201410328301', 'application No. 1501241', 'Application No. 1412306', 'Application No. 1501241']

US9853931B2 - Predictive forwarding of notification data - Google Patents
Predictive forwarding of notification data Download PDF
US9853931B2
US9853931B2 US15/183,142 US201615183142A US9853931B2 US 9853931 B2 US9853931 B2 US 9853931B2 US 201615183142 A US201615183142 A US 201615183142A US 9853931 B2 US9853931 B2 US 9853931B2
US15/183,142
US20160301649A1 (en
2013-12-10 Priority to US14/102,124 priority Critical patent/US8738723B1/en
2014-04-02 Priority to US14/242,917 priority patent/US9407591B2/en
2016-06-15 Application filed by Google LLC filed Critical Google LLC
2016-06-15 Priority to US15/183,142 priority patent/US9853931B2/en
2016-06-15 Assigned to GOOGLE INC. reassignment GOOGLE INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: FAABORG, ALEXANDER, ROBISON, AUSTIN
2016-10-13 Publication of US20160301649A1 publication Critical patent/US20160301649A1/en
2017-12-26 Publication of US9853931B2 publication Critical patent/US9853931B2/en
A computing system is described that receives notification data associated with a user, and responsive to determining that the user is interacting with a first computing device at a first particular time, outputs, for transmission to the first computing device, the notification data. The computing system determines that the user is interacting with a second computing device at a second particular time, and responsive to determining, based on the second particular time being less than a threshold amount of time after the first particular time, that the user did not view the notification data at the first computing device, outputs, for transmission to the second computing device, the notification data.
This application is a Continuation of application Ser. No. 14/242,917, filed on Apr. 2, 2014, which is a Continuation of application Ser. No. 14/102,124, filed on Dec. 10, 2013, the entire contents of each of which are hereby incorporated by reference.
A user may interact with a computing device that can receive notification data associated with the user. For instance, a user may interact with a desktop computer which can receive notification data associated with an instant messaging account of the user. While the user interacts with the desktop computer, the desktop computer may receive notification data indicating that the instant messaging account of the user has received a new message. The desktop computer may output a visual, audible, and/or haptic type alert to indicate to the user that the new instant message was received. However, in some instances the user may stop interacting with the desktop computer (e.g., exit the location of the desktop computer, travel beyond the physical range of an output device, or otherwise shift his or her focus away from the desktop computer) and thus miss the notification of the new instant message by failing to hear, see, and/or feel the alert.
In one example, the disclosure is directed to a method that includes receiving, by a computing system, notification data associated with a user. Responsive to determining that the user is interacting with a first computing device at a first particular time, the method further includes outputting, by the computing system and for transmission to the first computing device, the notification data. The method further includes determining, by the computing system, that the user is interacting with a second computing device at a second particular time, and responsive to determining, based on the second particular time being less than a threshold amount of time after the first particular time, that the user did not view the notification data at the first computing device, outputting, by the computing system and for transmission to the second computing device, the notification data.
In another example, the disclosure is directed to a computing system that includes at least one processor and at least one module operable by the at least one processor to receive notification data associated with a user, and responsive to determining that the user is interacting with a first computing device at a first particular time, output, for transmission to the first computing device, the notification data. The at least one module is further operable by the at least one processor to determine that the user is interacting with a second computing device at a second particular time, and responsive to determining, based on the second particular time being less than a threshold amount of time after the first particular time, that the user did not view the notification data at the first computing device, output, for transmission to the second computing device, the notification data.
In another example, the disclosure is directed to a computer readable storage medium including instructions that, when executed, configure one or more processors of a computing system to receive notification data associated with a user, and responsive to determining that the user is interacting with a first computing device at a first particular time, output, for transmission to the first computing device, the notification data. The computer readable storage medium includes further instructions that when executed configure the one or more processors of the computing system to determine that the user is interacting with a second computing device at a second particular time, and responsive to determining, based on the second particular time being less than a threshold amount of time after the first particular time, that the user did not view the notification data at the first computing device, output, for transmission to the second computing device, the notification data.
FIG. 1 is a conceptual diagram illustrating an example computing system that outputs notification data to one or more example computing devices, in accordance with one or more aspects of the present disclosure.
FIG. 2 is a block diagram illustrating an example computing system configured to output notification data, in accordance with one or more aspects of the present disclosure.
FIGS. 4 and 5 are flowcharts illustrating example operations of an example computing system configured to output notification data to one or more example computing devices, in accordance with one or more aspects of the present disclosure.
In general, techniques of this disclosure may enable a computing system to automatically determine whether or not to output notification data for transmission to a second computing device after previously outputting the notification data for transmission to a first computing device, thereby increasing a likelihood that a user associated with the notification data will receive (e.g., hear, view, and/or feel) an alert based thereon. For example, a user associated with notification data may switch between multiple computing devices such as a desktop or laptop computer, a mobile phone, a wearable computing device (e.g., a computerized watch, computerized eyeglasses, etc.) a tablet, and the like. A computing system (such as a server or cloud based computing platform) may manage transmission of notification data associated with the user across each computing device with which the user may interact. At an earlier particular time, the computing system may determine that the user associated with particular notification data is likely currently interacting with (e.g., is in proximity to and/or using) a first computing device and thus output the notification data for transmission to the first computing device (e.g., to cause the first computing device to output an alert based on the notification data).
At a later particular time, the computing system may determine that the user associated with the notification data is interacting with a second computing device (such as a wearable computing device or a mobile phone) and/or is otherwise no longer interacting with the first computing device, and as a result, may have not viewed, heard, and/or felt) an alert provided by first computing device at the first particular time. To improve a likelihood that the user associated with the notification data obtains the information associated with the notification data, the computing system may responsively output the notification data for transmission to the second computing device for output of a subsequent alert based on the notification data. In this manner, the computing system may cause notification data to “chase” the user from one device to another device (e.g., by outputting notification data for transmission across various computing devices) until the computing system determines that the user likely viewed, heard, and/or felt an alert based on the notification data.
FIG. 1 is a conceptual diagram illustrating an example computing system 1 that outputs notification data to example computing devices 10A-10N, in accordance with one or more aspects of the present disclosure. Computing system 1 of FIG. 1 is an example computing system that includes computing devices 10A-10N (collectively, “computing devices 10”), information server system 60, and network 30.
Network 30 represents any public or private communication network, for instance, a cellular, Wi-Fi, and/or other type of network for transmitting data between computing devices. Computing devices 10 and information server system 60 may send and receive data across network 30 using any suitable communication techniques. For example, computing device 10A may be operatively coupled to network 30 using network link 32A, and computing device 10N may be operatively coupled to network 30 using network link 32N. Information server system 60 may be operatively coupled to network 30 by network link 32X. Network 30 may include network hubs, network switches, network routers, etc., that are operatively inter-coupled thereby providing for the exchange of information between computing devices 10 and information server system 60. In some examples, network links 32A-32N, and 32X (collectively, “network links 32”) may be Ethernet, ATM or other network connections. Such connections may be wireless and/or wired connections.
In the example of FIG. 1, computing device 10A is a desktop computer and computing device 10N is a wearable computing device (e.g., a computerized watch, computerized eyewear, computerized gloves, etc.). However, in other examples, computing devices 10 may be any combination of tablet computers, mobile phones, personal digital assistants (PDA), laptop computers, gaming systems, media players, e-book readers, television platforms, automobile navigation systems, or any other types of mobile and/or non-mobile computing devices that are configured to receive notification data from information server system 60 and output an alert or otherwise provide an indication of the receipt of notification data.
As shown in FIG. 1, computing devices 10A-N each include respective user interface devices (UID) 12A-12N (collectively, “UIDs 12”). UIDs 12 of computing devices 10 may function as respective input and/or output devices for computing devices 10. UIDs 12 may be implemented using various technologies. For instance, UIDs 12 may function as input devices using presence-sensitive input screens, such as resistive touchscreens, surface acoustic wave touchscreens, capacitive touchscreens, projective capacitance touchscreens, pressure sensitive screens, acoustic pulse recognition touchscreens, or another presence-sensitive display technology. UIDs 12 may function as output (e.g., display) devices using any one or more display devices, such as liquid crystal displays (LCD), dot matrix displays, light emitting diode (LED) displays, organic light-emitting diode (OLED) displays, e-ink, or similar monochrome or color displays capable of outputting visible information to a user of computing devices 10.
Computing devices 10 may each include corresponding user interface (“UI”) modules 20A-20N (collectively, “UI modules 20”), notification client modules 22A-22N (collectively, “notification client modules 22”), and context information modules 24A-24N (collectively, “context information modules 24”). Modules 20, 22, and 24 may perform operations described using software, hardware, firmware, or a mixture of hardware, software, and firmware residing in and/or executing at respective computing devices 10. Computing devices 10 may each execute respective modules 20, 22, and 24 with one or more processors. Computing devices 10 may execute respective modules 20, 22, and 24 as one or more virtual machines executing on underlying hardware of computing devices 10. Modules 20, 22, and 24 may execute as one or more services or components of operating systems or computing platforms of computing devices 10. Modules 20, 22, and 24 may execute as one or more executable programs at application layers of computing platforms of computing devices 10. UIDs 12 and modules 20, 22, and 24 may be otherwise arranged remotely to and remotely accessible to respective computing devices 10, for instance, as one or more network services operating at network 30 in a network cloud.
Notification client modules 22 may perform functions associated with receiving, managing, and otherwise handling at least a portion of the notification data generated and/or received by platforms, applications, and services executing at computing devices 10. Notification client modules 22 may cause UI modules 20 to output alerts (e.g., an audible alert, a visual alert, a vibration, etc.) to indicate the receipt of notification data by computing devices 10.
Notification client modules 22 may each receive notification data from notification host module 62 of information server system 60 and output the received notification data to a recipient platform, application, and/or service executing at computing devices 10. Notification client modules 22 may receive notification data generated by a respective platform, application, and/or service executing at computing devices 10, and output the received notification data to information server system 60 over links 32.
As used throughout the disclosure, the term “notification data” is used to describe various types of information that may indicate the occurrence of an event associated with various platforms, applications, and services executing within an execution environment at one or more computing devices, such as computing devices 10. For example, notification data may include, but is not limited to, information specifying an event such as: the receipt of a communication message (e.g., e-mail, instant message, SMS, etc.) by a messaging account associated with a user of computing devices 10, the receipt of information by a social networking account associated with a user of computing devices 10, a reminder of a calendar event (meetings, appointments, etc.) associated with a calendar account of a user of computing devices 10, information generated and/or received by a third-party application executing at computing devices 10, the transmittal and/or receipt of inter-component communications between two or more components of platforms, applications, and/or services executing at computing devices 10, etc.
UI modules 20 may cause corresponding UIDs 12 to output respective user interfaces 14A-14N (collectively, “user interfaces 14”) (or other example user interfaces) for display, as a user views output and provides input at the respective UIDs 12. The same user may interact with two or more user interfaces 14, at different times. For instance, while in proximity to a location (e.g., an office) of computing device 10A, a user may interact with user interface 14A (e.g., a desktop computing device user interface) and, while away from the location of computing device 10A, the user may interact with user interface 14N (e.g., a wearable computing device user interface).
UI modules 20 may interpret inputs detected at UIDs 12 (e.g., as a user provides one or more gestures at one or more locations of UIDs 12 at which user interfaces 14 are displayed). UI modules 20 may relay information about the inputs detected at UIDs 12 to one or more associated platforms, operating systems, applications, and/or services executing at computing devices 10, to cause computing devices 10 to perform functions.
UI modules 20 may each receive information and instructions from one or more associated platforms, operating systems, applications, and/or services executing at computing devices 10 (e.g., notification client modules 22, etc.) for generating user interfaces 14. In addition, UI modules 20 may each act as respective intermediaries between the one or more associated platforms, operating systems, applications, and/or services executing at each of computing devices 10 and various output devices of computing devices 10 (e.g., speakers, LED indicators, audio or electrostatic haptic output device, etc.) to produce output (e.g., a graphic, a flash of light, a sound, a haptic response, etc.) with computing devices 10.
For instance, user interface 14A is one example graphical user interface for presenting one or more graphical indications of notification data received by computing device 10A. User interface 14A includes graphical indication 16A which represents a visual alert indicative of the receipt, by computing device 10A, of notification data associated with an instant message. User interface 14N is an example graphical user interface for presenting one or more graphical indications of notification data received by computing device 10N. User interface 14N includes graphical indication 16N which represents a visual alert indicative of the receipt, by computing device 10N, of notification data associated with an instant message. User interfaces 14 may include various other types of graphical indications such as a visual alert indicative of the receipt of notification data associated with an e-mail or other type of communication message, a social media network, a calendar appointment or reminder, etc.
UI modules 20 may receive graphical information (e.g., text data, images data, etc.) based on the notification data received by notification client modules 22 as input from notification client modules 22 along with instructions from notification client modules 22 for presenting the graphical information as graphical indications 16A-N (collectively, “graphical indications 16”) within user interfaces 14. Based on the graphical information and the instructions received from notification client modules 22, UI modules 20 may each cause corresponding UIDs 12 to present respective graphical indications 16 within respective user interfaces 14 to alert a user of a respective one of computing devices 10 of the receipt of notification data.
Context information modules 24 may each provide contextual information associated with computing devices 10 to prediction module 64 of information server system 60. Context information modules 24 may aggregate contextual information received and/or generated by respective onboard sensors, communication units and radios, input devices, and the like, of computing devices 10. Context information modules 24 may output respective contextual information to information server system 60 for use by prediction module 64 for determining whether a user associated with notification data is interacting with any of computing devices 10 at a particular time. For example, context information modules 24 may aggregate and output sensor information, communications and signal information, and/or information obtained by one or more input devices of computing devices 10 to prediction module 64.
As used throughout the disclosure, the term “contextual information” is used to describe information that can be used by a computing system, such as information server system 60, to determine location and/or degrees of movement associated with one or more computing devices, such as computing devices 10. In some examples, contextual information may include sensor information obtained by one or more sensors (e.g., gyroscopes, accelerometers, proximity sensors, etc.) of computing devices 10, radio transmission information obtained from one or more communication units and/or radios (e.g., global positioning system (GPS), cellular, Wi-Fi, etc.) of computing devices 10, information obtained by one or more input devices (e.g., cameras, microphones, keyboards, touchpads, mice, UIDs 12, etc.) of computing devices 10, and network/device identifier information (e.g., a network name, a device internet protocol address, etc.).
In addition to determining location and/or degrees of movement, contextual information can be used by a computing system, such as information server system 60, to determine the operating state of each of computing devices 10 at a particular time. In other words, based on the contextual information, prediction module 64 may determine whether any of computing devices 10 are active (e.g., powered-on, and/or receiving input), or inactive (e.g., powered-off, on standby, in sleep/screensaver mode, and/or not receiving input), and may use the determination about the operating state of computing devices 10 to determine whether a user is interacting with any of computing devices 10.
Information server system 60 represents any suitable remote computing system, such as one or more desktop computers, laptop computers, mainframes, servers, cloud computing systems, etc. capable of sending and receiving information (e.g., notification data) across network link 32X to network 30. In some examples, information server system represents a host server for a notification system service. One or more of computing devices 10 may access a notification service hosted by information server system 60 for transmitting and/or receiving notification data between platforms, applications, and services executing at the one or more computing devices. In some examples, information server system 60 represents a cloud computing system that provides notification services through network 30 to one or more of computing devices 10 that access the notification services via access to the cloud provided by information server system 60.
In the example of FIG. 1, information server system 60 includes notification host module 62 and prediction module 64. Modules 62 and 64 may perform operations described using software, hardware, firmware, or a mixture of hardware, software, and firmware residing in and/or executing at information server system 60. Information server system 60 may execute modules 62 and 64 with multiple processors or multiple devices. Information server system 60 may execute modules 62 and 64 as a virtual machines executing on underlying hardware. Modules 62 and 64 may execute as one or more services of an operating system or computing platform. Modules 62 and 64 may execute as one or more executable programs at an application layer of a computing platform.
Notification host module 62 may perform functions for routing notification data between one or more of computing devices 10 over network 30. Notification host module 62 may perform functions for hosting a notification service and also, functions for outputting notification data for transmission to one or more of computing devices 10. For example, notification host module 62 may receive notification data that indicates a message was received by an instant messaging account associated with computing devices 10 and may output the notification data for transmission across network 30 to any combination of computing devices 10.
Each of computing devices 10 may receive notification data from notification host module 62 of information server system 60 via a respective one of network links 32. Each of computing devices 10 may output alerts based on notification data received from notification host module 62 to indicate the receipt of the notification data by a respective one of computing devices 10.
As described in further detail below, notification host module 62 may first output notification data for transmission to a particular one of computing devices 10 (i.e., computing device 10A or computing device 10N), depending on which one of computing devices 10 that notification host module 62 determines is most likely being used by and “has the focus of” a user associated with the notification data. After outputting the notification data for transmission to the one of computing devices 10 that is most likely being used by the user, notification host module 62 may determine that the user may not have received (e.g., viewed, heard, and/or felt) an alert based on the notification data or otherwise obtained the information associated with the notification data. Notification host module 62 may determine a different one of computing devices 10 that is most likely being used by and has the attention of the user associated with the notification data, and may output the notification data for transmission to the different one of computing devices 10. By outputting the notification data a second time, notification host module 62 may increase the likelihood that the user associated with the notification data receives (e.g., views, hears, and/or feels) an alert based on the notification data or otherwise obtains the information associated with the notification data.
Prediction module 64 may perform functions for predicting which one of computing devices 10 is currently being used by a user associated with notification data. In general, prediction module 64 of information server system 60 may receive “contextual information” from each of computing devices 10 and, based on the contextual information, determine whether a user associated with notification data is or is not interacting with any of computing devices 10. Predication module 64 may respond to queries (e.g., from notification host module 62) requesting information indicating which one of computing devices 10 is more likely being used by a user associated with notification data.
Prediction module 64 may receive contextual information from each of computing devices 10 and determine locations associated with each of computing devices 10 based on the contextual information. For instance, prediction module 64 may receive GPS, cellular, and/or Wi-Fi data from one or more of computing devices 10 and determine respective coordinate locations of the one or more of computing devices 10. In some examples, prediction module 64 may determine relative locations (e.g., an address, a place, a country, a city, a type of building, a business, etc.) of computing devices 10 and/or determine locations of computing devices 10 based on signal triangulation (e.g., a technique for determining a single location based on relative position data received from two or more signals). In other examples, prediction module 64 may determine a location of one of computing devices 10 based on a network identifier associated with the one of computing devices 10.
Prediction module 64 may determine a degree of movement associated with each of computing devices 10 based on the contextual information received from any of computing devices 10. For example, prediction module 64 may determine a speed, acceleration, direction, and/or orientation of any of computing devices 10 based on sensor data received from computing devices 10 and/or based on location data associated with computing devices 10. For instance, prediction module 64 may track the locations of computing devices 10 over time and determine speed, direction, and/or acceleration of computing devices 10 based on changes in the locations of computing devices 10.
Based on the contextual information received from computing devices 10, prediction module 64 may determine a likelihood (e.g., a probability) of whether a user associated with notification data is interacting with any of computing devices 10 at a particular time. For example, prediction module 64 may use machine learning and/or other prediction algorithms and techniques, to determine a probability, based on the contextual information, that a user associated with notification data is interacting with any of computing devices 10 at a particular time. In other words, prediction module 64 may provide the contextual information as one or more signal inputs to a machine learning algorithm and receive as output, a probability indicating whether any of computing devices 10 is likely being used. Prediction module 64 may cause notification host module 62 to output notification data for transmission to the one of computing devices 10 that is more likely being used by the user at a particular time (e.g., in response to a query received from notification host module 62 for information indicating which one of computing devices 10 is more likely being used by the user at a particular time).
In some examples, information server system 60 may receive contextual information from computing devices 10 in substantially real-time. In other examples, information server system 60 may receive contextual information from computing devices 10 at various earlier times and use the earlier information to learn and produce rules for discerning which one of computing devices 10 is being used by a user associated with notification data, at a later time.
Prediction module 64 of information server system 60, and context information modules 24 of computing devices 10 may only collect or make use of information associated with a user of computing devices 10 after providing the user with an opportunity to provide input at computing devices 10 to control whether programs or features of information server system 60 and computing devices 10 can collect and make use of user information (e.g., information about a user's current location, current speed, etc.), or to dictate whether and/or how information server system 60 and computing devices 10 may receive content that may be relevant to the user. In addition, certain data may be treated in one or more ways before it is stored or used by information server system 60 and computing devices 10, so that personally-identifiable information is removed.
In operation, information server system 60 may receive notification data associated with a user. For example, notification host module 62 may receive notification data associated with a user of computing devices 10 indicating that a textual message was received by an instant messaging account of the user.
Responsive to determining that the user is interacting with computing device 10A at an earlier particular time, information server system 60 may output the notification data for transmission to computing device 10A. For instance, notification module 62 may determine, based on information provided by prediction module 64, that contextual information received from computing devices 10 indicates that computing device 10A is being used by the user associated with the notification data. Notification host module 62 may output the notification data over links 32 to computing device 10A.
Information server system 60 may determine that the user associated with the notification data is interacting with computing device 10N at a later particular time. For example, at a later particular time (i.e., after the earlier time at which information server system 60 determined that the user was interacting with computing device 10A), notification host module 62 may receive information from prediction module 64 that indicates computing device 10A is not likely being used by the user and/or computing device 10N is likely being used by the user.
Responsive to determining, based on the later particular time being less than a threshold amount of time after the earlier particular time, that the user did not view the notification data at computing device 10A. Notification host module 62 may determine whether the amount of time between the earlier time and the later time is less than a threshold amount of time (e.g., less than one minute, thirty seconds, etc.) that represents a period of time during which the user may have changed his or her focus away from computing device 10A and missed or otherwise been prevented from being notified by notification client module 22 of computing device 10A of the receipt of the notification data. Information server system 60 may output, for transmission to computing device 10N, the notification data and notification client module 22 of computing device 10N may cause UI module 22N to output an alert based on the notification data (e.g., graphical indication 16N at UID 12).
In this manner, a computing system according to the techniques of this disclosure may cause notification data to “chase” a user associated with the notification data from a first computing device to a second computing device if the computing system determines that a user may have missed the notification data at the first computing device. The computing system may determine that the first computing device received notification data during a period of time when the user associated with the notification data may have been changing focus and shifting his or her attention away from the first computing device. The computing system may “chase the user” and automatically output the notification data to a second computing device that is more likely being used by the user at the current time.
By outputting notification data to a second computing device in this way, the computing system may increase the likelihood that the user receives notifications alerts and/or obtains information associated with notification data. Additionally, the computing system may decrease the likelihood that the user misses notification alerts and/or notification based information. The various computing devices that receive notification data from the computing system in this way may receive fewer inputs from a user interacting with the various computing devices (e.g., to obtain information using a computing device that was already provided as notification data to a different computing device). In turn, the various computing devices may receive fewer false inputs from the user and, as such, the various computing devices and computing system may perform fewer operations and consume less electrical power.
In some examples, prediction module 64 may determine the likelihood (e.g., a probability) of whether the user associated with notification data is interacting with any of computing devices 10 at a particular time based on contextual information received from computing devices 10 that includes input data detected by one or more input devices (e.g., mice, trackballs, touchpads, etc.) of computing device 10. Prediction module 64 may identify specific patterns associated with the input data and identify a series of specific user interactions that indicate whether the user is interacting with any of computing devices 10 based on the identified patterns. Prediction module 64 may use sensor data associated with the contextual information to determine whether any of the identified patterns represent a change in user behavior and indicate that the user is interacting, or is no longer interacting, with computing devices 10.
For example, based on the contextual information, prediction module 64 may determine that input data received from computing device 10A indicates that a user is moving a mouse type input device (e.g., trackball, touch pad, etc.) to interact with computing device 10A (e.g., user interface 14A) at an earlier particular time. At a later particular time, prediction module 64 may determine that the input data indicates that the user is not using the mouse type device.
The machine learning and/or other prediction algorithms and techniques, of prediction module 64 may use the input data received from computing device 10A at the earlier and later particular times, along with accelerometer sensor type data received from computing device 10N at the later and earlier particular times to determine that the user is no longer interacting with computing devices 10A at the later particular time. For example, accelerometer data received from computing device 10N may indicate that the user is moving at the later particular time. Prediction module 64 may determine that the likelihood that the user is interacting with computing device 10A at the later particular time, when the user is moving, and when the user is not using a mouse, is very low, and as such, determine that the user is not interacting with computing device 10A at the later particular time. In other words, prediction module 64 may predict that the user is sitting at computing device 10A at an earlier particular time (as the user is moving the mouse) and determine that the user has gotten up and is no longer sitting at computing device 10A at the later particular time.
In some examples, input data contained within the contextual information received by prediction module 64 may include video data (e.g., received by a webcam of computing device 10A). Prediction module 64 may determine that a user is interacting with any of computing devices 10 based on face and/or retina imaging matching techniques applied to the video data received from a webcam of computing device 10A. In other words, prediction module 64 may determine that an image associated with video data received by a camera of computing device 10A matches or appears similar to a human face, and/or a particular user's face if the user were to view a screen or monitor associated with computing device 10A. Prediction module 64 may determine that the user is interacting with computing device 10A when a human and/or user face is identified by prediction module 64 within a frame of the video data at an earlier particular time. At a later particular time, prediction module 64 may determine that the user is no longer interacting with computing device 10A when the video data does not include a frame of video data that has a human and/or user face.
In some examples, sensor data contained within contextual information associated with computing devices 10 may alone be enough for prediction module 64 to determine whether a user is interacting with any of computing devices 10. For instance, accelerometer data and/or gyro data received from computing device 10N may indicate a position and orientation of computing device 10N and prediction module 64 may determine whether computing device 10N is being held and/or worn, or not being held and/or not being worn based on the position and orientation information. Prediction module 64 may determine, based on accelerometer data, that computing device 10N is resting on a flat, surface and/or determine that computing device 10N is not likely being worn. Or if computing device 10N is a tablet, prediction module 64 may determine, based on accelerometer data, that computing device 10N is lying flat on a surface. Prediction module 64 may determine a high degree of likelihood that computing device 10N is not being used at a particular time for instance, if computing device 10N is wearable and not being worn, or if computing device 10N is a tablet type device and lying flat on a surface (instead of being held by the user).
FIG. 2 is a block diagram illustrating an example computing system configured to output notification data, in accordance with one or more aspects of the present disclosure. Information server system 60 of FIG. 2 is described below within the context of FIG. 1. FIG. 2 illustrates only one particular example of information server system 60, and many other examples of information server system 60 may be used in other instances and may include a subset of the components included in example information server system 60 or may include additional components not shown in FIG. 2.
As shown in the example of FIG. 2, information server system 60 includes one or more processors 70, one or more communication units 72, and one or more storage devices 74. Storage devices 74 of information server system 60 include notification host module 62 and prediction module 64. Storage devices 74 of information server system 60 further include notification information data store 68A, and device context information data store 68B (collectively, “data stores 68”). Communication channels 80 may interconnect each of the components 70, 72, and 74 for inter-component communications (physically, communicatively, and/or operatively). In some examples, communication channels 80 may include a system bus, a network connection, an inter-process communication data structure, or any other method for communicating data.
One or more communication units 72 of information server system 60 may communicate with external computing devices, such as computing device 10, by transmitting and/or receiving network signals on one or more networks, such as network 30. For example, information server system 60 may use communication unit 72 to transmit and/or receive radio signals via links 32 to exchange information with computing devices 10. Examples of communication unit 72 include a network interface card (e.g. such as an Ethernet card), an optical transceiver, a radio frequency transceiver, a GPS receiver, or any other type of device that can send and/or receive information. Other examples of communication units 72 may include short wave radios, cellular data radios, wireless Ethernet network radios, as well as universal serial bus (USB) controllers.
One or more storage devices 74 within information server system 60 may store information for processing during operation of information server system 60 (e.g., information server system 60 may store data accessed by modules 62 and 64 during execution at information server system 60). In some examples, storage devices 74 are a temporary memory, meaning that a primary purpose of storage devices 74 is not long-term storage. Storage devices 74 on information server system 60 may be configured for short-term storage of information as volatile memory and therefore not retain stored contents if powered off. Examples of volatile memories include random access memories (RAM), dynamic random access memories (DRAM), static random access memories (SRAM), and other forms of volatile memories known in the art.
Storage devices 74, in some examples, also include one or more computer-readable storage media. Storage devices 74 may be configured to store larger amounts of information than volatile memory. Storage devices 74 may further be configured for long-term storage of information as non-volatile memory space and retain information after power on/off cycles. Examples of non-volatile memories include magnetic hard discs, optical discs, floppy discs, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories. Storage devices 74 may store program instructions and/or data associated with modules 62 and 64.
One or more processors 70 may implement functionality and/or execute instructions within information server system 60. For example, processors 70 on information server system 60 may receive and execute instructions stored by storage devices 74 that execute the functionality of modules 62 and 64. These instructions executed by processors 70 may cause information server system 60 to store information, within storage devices 74 during program execution. Processors 70 may execute instructions of modules 62 and 64 to provide fitness and navigation information to one or more computing devices (e.g., computing devices 10 of FIG. 1). That is, modules 62 and 64 may be operable by processors 70 to perform various actions or functions of information server system 60.
Data stores 68 represent any suitable storage medium for storing data related to notification data and contextual information. For example, notification information data store 68A may store notification data received by notification host module 62 and device context information data store 68B may store contextual information received by prediction module 64 and associated with, and/or received from, one or more of computing devices 10.
The notification data stored at data store 68A may be sorted, organized, and/or accessed by notification host module 62 according to various attributes associated with the notification data. For instance, in addition to including content (e.g., information associated with the notification data such as text, video, images, etc. associated with a message, calendar reminder, etc.), each portion of notification data received by information server system may include one or more attributes that notification host module 62 uses for determining when to output the notification data and/or which one or more of computing devices 10 of network 30 to route or deliver the notification data.
One attribute may include a user identifier, such as a user name, account name, telephone number, network identifier, e-mail address, etc. that indicates a user recipient associated with the notification data for which computing devices 10 may output an alert based on the notification data and/or otherwise provide access to information associated with the notification data. Notification host module 62 may store received notification data at data store 68A according to the user identifier and may retrieve notification data from data store 68A by querying notification data based on the user identifier.
In some examples, the notification data may include a sender identifier that indicates an origin of the notification data. For example, e-mail or instant message notification data may include an attribute that specifies the e-mail address or instant message account of a user “sender” associated with the notification data. In other words, the sender identifier may indicate an account name associated with a person that typed the e-mail or instant message associated with the notification data.
In some examples, the notification data at notification information data store 68A may include a “priority level” attribute assigned to the notification data by the notification service and/or the origin of the notification data. For example, the computing device and/or system that generates notification data may include a priority attribute associated with the data and determined and/or assigned to the notification data by the sender, so that the sender, by assigning a priority level, can specify a level of importance of the notification data as compared to other types of notification data. The priority level may specify a degree of importance associated with the notification data for causing notification host module 62 to prioritize an output of the notification data. Notification host module 62 may compare a priority level of notification data to one or more priority thresholds to determine when to output notification data to computing devices 10. For instance, a priority level that exceeds a priority threshold may cause notification host module 62 to output the notification data within one particular period or duration of time (e.g., one minute after receipt at information server system 60, thirty seconds after receipt, etc.) and a priority level that does not exceed the threshold may cause notification host module 62 to delay outputting the notification data until a later time (e.g., five minutes after receipt at information server system 60, one hour after receipt, etc.).
In some examples, the notification data at notification information data store 68A may include a “time sensitivity” attribute assigned to the notification data by the notification service and/or the origin of the notification data. The time sensitivity attribute may specify a latest particular time at which to deliver the notification data or otherwise specify an amount of time for which the information associated with the notification data is most relevant to a user associated with the notification data. In some examples, the time sensitivity attribute may mark or tag the notification data with an indication that the information associated with the notification data is time dependent and/or time sensitive. Notification host module 62 may compare the time sensitivity attribute of notification data to a threshold to determine whether the notification data is time sensitive (e.g., if the attribute satisfies the threshold) or whether delivery of the notification data (e.g., via an alert based on the notification data) is not required by a particular time (e.g., if the attribute does not satisfy the threshold).
The contextual information stored at data store 68A may be organized according to one or more individual and/or groups of computing devices 10 and/or according to one or more individual users associated with computing devices 10. For instance, prediction module 64 may receive contextual information from computing devices 10 and categorize the contextual information at device context information data store 68A by grouping the contextual information from each one of computing devices 10 at separate, respective portions or locations of data store 68A. Each separate, respective portion or location of data store 68A may have a label associated with one or more computing devices 10 and may also have a label associated with a user. Prediction module 64 may retrieve contextual information from data store 68A by performing queries for the user label and/or computing device label to obtain the contextual information associated with a specific user and/or specific one or more of computing devices 10.
In some examples, prediction module 64 may collect (i.e., receive) and store (e.g., at device context information data store 68B) contextual information from a number of different computing devices 10 over a period of time, but only if information server system 60 receives explicit permission from the various users of computing devices 10 to receive and analyze the information. For example, in situations discussed herein in which information server system 60 may collect or may make use of personal information about any user (e.g., contextual information), the user may be provided with an opportunity to control whether programs or features of the computing system can collect user information (e.g., a user's current location, a user's speed, a user's direction, etc.). In addition, certain data may be treated in one or more ways before it is stored or used by the computing device, so that personally identifiable information is removed. For example, a user's calendar information, preferences, geographic locations, etc., may be generalized so that an exact user's calendar, preference, and geographic location cannot be determined. Thus, the user may have control over how information is collected about the user and used by information server system 60.
In operation, notification host module 62 may receive notification data associated with a user of computing devices 10 indicating that a textual message was received by an instant messaging account of the user. Notification host module may store at least a portion of the notification data at data store 68A. Notification host module 62 may query prediction module 64 to obtain information, from prediction module 64, indicating which of computing devices 10 that is more likely being used by the user at the particular time.
Prediction module 64 may access device context information data store 68B for contextual information received from one or more computing devices 10 to determine which one of computing devices 10 is most likely being used by the user associated with the notification data, at the particular time. Prediction module 64 may perform a query at data store 68B using a user identifier associated with the notification data to retrieve contextual information associated with the one or more computing devices of computing devices 10 that may be used by the user. Based on the contextual information retrieved from data store 68B, prediction module 64 may determine which of computing devices 10 is most likely being used at the particular time and may further determine which one of computing devices 10 may be better suited for receiving the notification data from notification host module 62 and for gaining the attention of the user by outputting an alert based on the instant message notification data, at the particular time.
For instance, prediction module 64 may determine that contextual information from context information modules 24 indicates that computing device 10A is powered-on and operating in an active state and that one or more input devices of computing device 10A are actively receiving input from the user (e.g., as a user types at a keyboard and/or provides touch input at UID 12A). Based on the contextual information from context information modules 24, prediction module 64 may determine a probability (e.g., using machine learning, or other types or rules based predictive algorithms and techniques) that computing device 10A is being used at the particular time. Responsive to determining that the probability satisfies a probability threshold (e.g., fifty percent), prediction module 64 may determine that the user associated with the notification data is using computing device 10A at the particular time.
In some examples, prediction module 64 may use contextual information received from one of computing devices 10 and stored at data store 68B to determine that a user is using the other one of computing device 10. For example, based on contextual information received from computing device 10A, prediction module 64 may determine that computing device 10A is a desktop computer that is active and stationary at a particular location. Prediction module 64 may determine the user of computing device 10A is likely using and is focused on computing device 10A.
Based on contextual information received from computing device 10N, prediction module 64 may determine that computing device 10N is an inactive, wearable computing device, which is also stationary at the particular location. Although, computing device 10N is inactive, prediction module 64 may receive contextual information from computing device 10N that indicates computing device 10N is being actively worn (e.g., based on sensor information indicating a galvanic skin response, temperature, orientation, and or other movement, that indicates that computing device 10N is on a person's wrist). Prediction module 64 may determine the user of computing device 10A is not likely using and/or focused on computing device 10N.
Responsive to determining that the user is interacting with computing device 10A at an earlier particular time, information server system 60 may output the notification data for transmission to computing device 10A. For example, notification host module 62 may receive information from prediction module 64 that indicates that computing devices 10A has a higher probability of being used at the earlier particular time than the probability of computing device 10N being used at the earlier particular time. Notification host module 62 may sent the notification data over communication channels 50 to communication units 72 that may output the notification data over links 32 to notification client module 22A of computing device 10A. Notification client module 22A of computing device 10A may output graphical information and/or instructions to command UI module 20A to cause UID 12A to output an alert (e.g., graphical indication 16A, and the like) based on the notification data.
To increase a likelihood that the user associated with the notification data obtains information associated with the notification data and/or receives an alert based on the notification data, notification host module 62 may determine whether the user associated with the notification data continues to interact with computing device 10A at a later particular time (e.g., after the earlier particular time at which notification host module 62 first outputs the notification data for transmission to computing device 10A). If notification host module 62 determines that the user is interacting with computing device 10N at the later particular time, notification host module 62 may infer that the user may have missed the notification data during the period of time (e.g., between the earlier particular time and the later particular time) at which the user changed his or her focus and stopped using computing device 10A and/or began using computing device 10N.
Information server system 60 may determine that the user is interacting with computing device 10N at a later particular time. For example, at the later particular time, notification host module 62 may query prediction module 64 for information indicating which one of computing devices 10 is more likely being used by the user and receive an indication that computing device 10A is not likely being used by the user and/or computing device 10N is likely being used by the user.
Responsive to determining, based on the later particular time being less than a threshold amount of time after the earlier particular time, that the user did not view the notification data at computing device 10A, information server system 60 may output, for transmission to computing device 10N, the notification data. In other words, notification host module 62 may determine whether the user associated with the notification data likely missed the notification data (e.g., an alert provided by computing device 10A) by comparing the earlier time at which the notification data was first output for transmission to computing device 10A to the later time at which the user may have begun using computing device 10A and/or may have stopped using computing device 10N. Notification host module 62 may determine whether the amount of time between the earlier time and the later time is less than a threshold amount of time (e.g., less than one minute, thirty seconds, etc.). The threshold may represent a period of time during which the user may have changed his or her focus away from computing device 10A and missed or otherwise been prevented from being notified by notification client module 22 of computing device 10A of the receipt of the notification data.
Notification host module 62 may output the notification data to computing device 10N. Notification client module 22N of computing device 10N may cause UI module 22N to output an alert based on the notification data (e.g., graphical indication 16N at UID 12N).
In some examples, information server system 60 may output notification data for transmission to a second computing device after first outputting the notification data for transmission to a first computing device, in response to determining that a priority level of the notification data satisfies a threshold. In other words, notification host module 62 may determine whether a priority level of notification data satisfies a priority threshold before determining whether or not to “chase” a user with notification data by outputting the notification data at a second one of computing devices 10 at a later particular times when notification host module 62 determines that the user did or did not likely receive (e.g., viewed, heard, and/or felt) an alert based on notification data or otherwise obtained information associated with the notification data when the notification data was output for transmission to a first one of computing devices 10, at an earlier particular time. If a priority level of the notification data satisfies a threshold (e.g., for indicating more important notification data), notification host module 62 may query prediction module 64 for information indicating which one of computing devices 10 that is most likely being used by the user and may cause communication modules 72 to output the notification data for transmission to that most likely one of computing devices 10.
In some examples, information server system 60 may output notification data for transmission to a second computing device after first outputting the notification data for transmission to a first computing device, in response to determining that the notification data includes time sensitive information. In other words, notification host module 62 may determine whether a time sensitivity attribute of received notification data indicates that the information associated with the notification data will expire at a particular time, before determining whether or not to “chase” the user associated with the notification data and output the notification data at second, third, fourth, etc. computing devices 10 at later particular times until notification host module 62 determines that the user likely received (e.g., viewed, heard, and/or felt) an alert based on notification data or otherwise obtained the information associated with the notification data. If notification host module 62 determines that the information associated with the notification data is time sensitive (e.g., by analyzing a time sensitivity attribute), notification host module 62 may query prediction module 64 for information indicating which one of computing devices 10 that is most likely being used by the user and may cause communication modules 72 to output the notification data for transmission to that most likely one of computing devices 10.
In some examples, information server system 60 may output notification data for transmission to a second computing device after first outputting the notification data for transmission to a first computing device, in response to determining that a sender identifier associated with the notification data corresponds to an entry of an electronic address book or contact list. In other words, notification host module 62 may chase a user associated within notification data after determining that the notification data is associated with a contact of a contact list associated with the user. For example, notification host module 62 access or store information associated with an address book or contact list associated with a user (e.g., via a cloud service accessible to information server system 60). Notification host module 62 may correlate a sender identifier of notification data with one or more entries of an electronic address book or contact lists associated with a user to determine whether the sender identifier corresponds to an entry in the address book or contact list. Notification host module 62 may “chase” a user (e.g., by outputting the notification data to different one of computing devices 10 at a later time until notification host module 62 determines that the user associated with the notification data likely received an alert based on the notification data or otherwise obtained the information associated with the notification data) and output the notification data to a second computing device when the sender is contained within an electronic address book or contact list associated with the user and/or not chase the user when the sender is not contained within the address book or contact list.
FIG. 3 is a block diagram illustrating computing device 100 that outputs graphical content for display at a remote device, in accordance with one or more techniques of the present disclosure. Graphical content, generally, may include any visual information that may be output for display, such as text, images, a group of moving images, etc. The example shown in FIG. 3 includes a computing device 100, presence-sensitive display 101, communication unit 110, projector 120, projector screen 122, mobile device 126, and visual display device 130. Although shown for purposes of example in FIG. 1 as multiple stand-alone computing devices 10, a computing device such as one of computing devices 10, and computing device 100 may, generally, be any component or system that includes a processor or other suitable computing environment for executing software instructions and, for example, need not include a presence-sensitive display.
In other examples, such as illustrated previously by computing devices 10 in FIG. 1, a computing device may refer to a portable or mobile device such as mobile phones (including smart phones), laptop computers, computing watches, computing eye glasses, wearable computing devices, etc. In some examples, a computing device may be a desktop computers, tablet computers, smart television platforms, cameras, personal digital assistants (PDAs), servers, mainframes, etc.
In accordance with techniques of the disclosure, computing device 100 may be operatively coupled to visual display device 130 using external network 114. Computing device 100 may receive notification data as output from a computing system (e.g., information server system 60 of FIGS. 1 and 2). For instance, the computing system may determine that a user associated with the notification data is likely interacting with computing device 100 at a particular time and in response, output the notification data for transmission to computing device 100.
Responsive to receiving the notification data, computing device 100 may output an alert indicative of the receipt of the notification data. For examples, computing device 100 may output, for display, a graphical indication (e.g., graphical indication 16A) based on the notification data to visual display device 130. Computing device 100 may output, for display, the graphical indication based on the notification data via direct device communication 118 or external network 114 to display device 130. In some examples, display device 130 outputs the graphical indication for display to the user associated with the notification data and the user may, in turn, interact with computing device 100 by selecting or dismissing some or all of the displayed graphical indication based on the notification data. Computing device 100 and display device 130 may be configured such that selection of a particular element of the displayed graphical based on the notification data, may cause computing device 100 to obtain additional information (e.g., text of an instant message, etc.) associated with the notification data.
FIGS. 4 and 5 are flowcharts illustrating example operations of an example computing system configured to output notification data to one or more example computing devices, in accordance with one or more aspects of the present disclosure. The processes of FIGS. 4 and 5 may be performed by one or more processors of a computing system, such as information server system 60 illustrated in FIG. 1 and FIG. 2. For purposes of illustration, FIGS. 4 and 5 are described below within the context of information server system 60, computing devices 10, and system 1, illustrated in FIG. 1.
FIG. 4 illustrates that information server system 60 may receive notification data associated with a user (200). For example, information server system 60 may receive notification data indicative of the receipt of an e-mail message addressed to an e-mail account accessed by a user of computing devices 10.
Information server system 60 may determine that the user is interacting with a first computing device at a first particular time (210). For example, prediction module 64 may output information to notification host module 62 that indicates that the user of computing devices 10 is likely interacting with computing device 10A at a first particular time.
FIG. 4 shows that information server system 60 may output the notification data for transmission to the first computing device (220). For example, notification host module 62 may cause information server system 60 to output the notification data via links 32 to notification client module 22A of computing device 10A.
In some examples, information server system 60 may output the notification data for transmission to the first computing device (e.g., computing device 10A) in response to determining that the user is interacting with the first computing device at the first particular time and may not output the notification data for transmission to other computing devices 10, including the second computing device (e.g., computing device 10N), at the first particular time. In other words, information server system 60 may exclusively output the notification data for transmission to the single computing device (e.g., computing device 10) that information server system 60 determines is more likely being used at the first particular time.
Information server system 60 may determine that the user is interacting with a second computing device at a second particular time (230). For example, notification host module 62 may determine that the notification data is associated with a contact in an address book or contact list associated with the user, or that a priority level associated with the notification data exceeds a priority threshold, and/or that the notification data includes time sensitive information. To increase a likelihood that the user associated with the notification data receives an alert based on the notification data or otherwise obtains the notification data that information server system 60 output at the first (earlier) particular time, notification host module 62 may query prediction module 64 for an updated indication of whether the user is interacting with computing device 10A or not at a second (later) particular time. In other words, notification host module 62 may determine that the user did not view the notification data at the first computing device by determining that the user is not interacting with the first computing device at the second particular time. Notification host module 62 may receive information from prediction module 64 which indicates, that at the second particular time, the user is more likely interacting with computing device 10N and/or is not likely interacting with computing device 10A.
In some examples, information server system 60 may determine that the user did not view the notification data at the first computing device by first receiving contextual information associated with the second computing device, and determining, based on the contextual information, that the user is not interacting with the first computing device at the second particular time. In other words, prediction module 64 may receive contextual information from computing device 10N and determine, based on the contextual information received from computing device 10N, that the user is not using computing device 10A. Prediction module 64 may determine that a user is not interacting with one of computing devices 10, such as a desktop computer, based on contextual information (e.g., sensor information, etc.) received from a different one of computing devices 10, such as a computerized watch.
FIG. 4 shows information server system 60 may determine whether the second particular time is less than a threshold amount of time after the first particular time (240). For instance, notification host module 62 may determine that the second particular time is too close in time to the first particular time and indicates that the user may have missed an alert based on the notification data at computing device 10A (e.g., by shifting his or her focus away from computing device 10A viewing, hearing, and/or feeling an alert based on the notification data at computing device 10A).
If information server system 60 determines that the second particular time is less than the threshold amount of time after the first particular time, information server system 60 may output the notification data for transmission to the second computing device (250). For instance, information server system 60 may output the notification data for transmission to computing device 10N in an attempt to “chase” the user and cause one of computing devices 10 that is most likely being used by the user at the second particular time to output an alert based on the notification data.
In some examples, responsive to determining that the user is interacting with the first computing device at a third particular time after the second particular time, information server system 60 may output the notification data for transmission to computing device 10A. In other words, after outputting the notification data to computing device 10N, information server system 62 may determine that the user is likely interacting with computing device 10A again and may output the notification data via a second transmission to computing device 10A.
FIG. 5 illustrates further operations of information server system 60. For instance, FIG. 5 shows an additional example of operations of information server system 60 that may be performed by information server system 60 from step 250 of FIG. 4.
FIG. five shows information server system 60 may receive second notification data associated with the user (260). For example, information server system 60 may receive additional notification data, the additional notification data being indicative of the receipt of an instant message addressed to an instant message account accessed by the user of computing devices 10.
Information server system 60 may output the second notification data for transmission to the second computing device (270). For example, notification host module 62 may receive information from prediction module 64 that indicates the user associated with the notification data continues to interact with computing device 10N. Notification host module 62 may transmit the additional notification data to computing device 10N and cause notification client module 22N and UI module 20N to output an alert based on the additional notification data.
Information server system 60 may determine that the user is interacting with the first computing device at a third particular time after the first and second particular times (280). In other words, at a third particular time, after the first and second particular times, notification module 62 may receive information from prediction module 64 that indicates that the user has likely began interacting again with computing device 10A. Notification host module 62 may determine that the third particular time is less than the threshold amount of time after the second particular time and determine that the user may not have viewed, heard, and/or felt (e.g., received) an alert based on the additional notification data at computing device 10N.
Information server system 60 may output the second notification data received after the first particular time and prior to the third particular time for transmission to the first computing device (290). In other words, responsive to determining that the user is interacting with computing device 10A at a third particular time after the second particular time, information server system 62 may output, for transmission to computing device 10A, the additional notification data received by information server system 62 after the first particular time and prior to the third particular time. In this manner, information server system 62 may chase the user associated with notification data back and forth between computing devices 10. Information server system 62 may output first notification data (e.g., an e-mail based notification) at computing device 10A and then “chase” and output the first notification data at computing device 10N. After receiving additional notification data (e.g., an instant message based notification) information server system 62 may output the additional notification data for transmission to computing device 10N and then “chase” and output the additional notification data for transmission to computing device 10A.
In some examples, information server system 62 may determine whether a user is interacting with any of computing device 10 based on face detection techniques. For instance, computing device 10A may include a camera (e.g., a web camera, etc.). Information server system 62 may receive video data captured by the camera of computing device 10A and using facial recognition techniques and/or algorithms, determine whether or not an image of a human face is contained with the video data. In cases when a human face is detected within the video data, information server system 62 may determine that the user is interacting with computing device 10A. In cases when a human face is not detected within the video data, information server system 62 may determine that the user is not interacting with computing device 10A. In some examples, retina scanning or other image matching techniques may be used to determine whether an image associated with still image or motion video data received from any of computing devices 10 indicates whether or not a user is interacting with any of computing devices 10.
Clause 1. A method, comprising: receiving, by a computing system, notification data associated with a user; responsive to determining that the user is interacting with a first computing device at a first particular time, outputting, by the computing system and for transmission to the first computing device, the notification data; determining, by the computing system, that the user is interacting with a second computing device at a second particular time; and responsive to determining, based on the second particular time being less than a threshold amount of time after the first particular time, that the user did not view the notification data at the first computing device, outputting, by the computing system and for transmission to the second computing device, the notification data.
Clause 2. The method of clause 1, wherein determining that the user did not view the notification data at the first computing device further comprises determining that the user is not interacting with the first computing device at the second particular time.
Clause 3. The method of any of clauses 1-2, wherein the notification data is output for transmission to the first computing device in response to determining that the user is interacting with the first computing device at the first particular time and is not output for transmission to other computing devices, including the second computing device, at the first particular time.
Clause 4. The method of any of clauses 1-3, wherein outputting, by the computing system and for transmission to the second computing device, the notification data is further responsive to determining that a priority level of the notification data satisfies a threshold.
Clause 5. The method of any of clauses 1-4, wherein outputting, by the computing system and for transmission to the second computing device, the notification data is further responsive to determining that the notification data includes time sensitive information.
Clause 6. The method of any of clauses 1-5, wherein outputting, by the computing system and for transmission to the second computing device, the notification data is further responsive to determining that the notification data is associated with a contact of a contact list associated with the user.
Clause 7. The method of any of clauses 1-6, further comprising responsive to determining that the user is interacting with the first computing device at a third particular time after the second particular time, outputting, by the computing system, for transmission to the first computing device, the notification data.
Clause 8. The method of any of clauses 1-7, wherein the notification data comprises first notification data, the method further comprising responsive to determining that the user is interacting with the first computing device at a third particular time after the second particular time, outputting, for transmission to the first computing device, second notification data received by the computing system after the first particular time and prior to the third particular time.
Clause 9. The method of any of clauses 1-8, wherein determining that the user did not view the notification data at the first computing device further comprises: receiving, by the computing system, contextual information associated with the second computing device; and determining, by the computing system and based on the contextual information, that the user is not interacting with the first computing device at the second particular time.
Clause 10. A computing system comprising: at least one processor; and at least one module operable by the at least one processor to: receive notification data associated with a user; responsive to determining that the user is interacting with a first computing device at a first particular time, output, for transmission to the first computing device, the notification data; determine that the user is interacting with a second computing device at a second particular time; and responsive to determining, based on the second particular time being less than a threshold amount of time after the first particular time, that the user did not view the notification data at the first computing device, output, for transmission to the second computing device, the notification data.
Clause 11. The computing system of clause 10, wherein the at least one module is further operable by the at least one processor to determine that the user did not view the notification data at the first computing device by at least determining that the user is not interacting with the first computing device at the second particular time.
Clause 12. The computing system of any of clauses 10-11, wherein the at least one module is further operable by the at least one processor to output the notification data for transmission to the first computing device in response to determining that the user is interacting with the first computing device at the first particular time and to not output the notification data for transmission to other computing devices, including the second computing device, at the first particular time.
Clause 13. The computing system of any of clauses 10-12, wherein the at least one module is further operable by the at least one processor to output the notification data responsive to determining that the user did not view the notification data at the first computing device further responsive to determining that a priority level of the notification data satisfies a priority threshold.
Clause 14. The computing system of any of clauses 10-13, wherein the at least one module is further operable by the at least one processor to output the notification data responsive to determining that the user did not view the notification data at the first computing device and further responsive to determining that the notification data includes time sensitive information.
Clause 15. The computing system of any of clauses 10-14, wherein the at least one module is further operable by the at least one processor to output the notification data responsive to determining that the user did not view the notification data at the first computing device, and further responsive to determining that the notification data is associated with a contact of a contact list associated with the user.
Clause 16. The computing system of any of clauses 10-15, wherein the at least one module is further operable by the at least one processor to responsive to determining that the user is interacting with the first computing device at a third particular time after the second particular time, output, for transmission to the first computing device, the notification data.
Clause 17. The computing system of any of clauses 10-16, wherein the notification data comprises first notification data, and wherein the at least one module is further operable by the at least one processor to responsive to determining that the user is interacting with the first computing device at a third particular time after the second particular time, output, for transmission to the first computing device, second notification data received after the first particular time and prior to the third particular time.
Clause 18. The computing system of any of clauses 10-17, wherein the at least one module is further operable by the at least one processor to determine that the user did not view the notification data at the first computing device by at least receiving contextual information associated with the second computing device, and determining, based on the contextual information, that the user is not interacting with the first computing device at the second particular time.
Clause 19. The computing system of any of clauses 10-18, wherein the first computing device is a non-mobile computing device and the second computing device is a mobile computing device, wherein the at least one module is further operable by the at least one processor to determine that the user did not view the notification data at the first computing device based at least in part on contextual information associated with the second computing device, wherein the contextual information comprises sensor information of the second computing device.
Clause 20. A computer-readable storage medium comprising instructions that, when executed, configure one or more processors of a computing system to: receive notification data associated with a user; responsive to determining that the user is interacting with a first computing device at a first particular time, output, for transmission to the first computing device, the notification data; determine that the user is interacting with a second computing device at a second particular time; and responsive to determining, based on the second particular time being less than a threshold amount of time after the first particular time, that the user did not view the notification data at the first computing device, output, for transmission to the second computing device, the notification data.
Clause 21. A computing device comprising means for performing any of the methods of clauses 1-9.
Clause 22. A computer-readable storage medium encoded with instructions for causing one or more programmable processors to perform any of the methods recited by clauses 1-9.
outputting, by the mobile computing device, at a first particular time, an indication of notification data associated with the user;
determining, by the mobile computing device, that the user is not interacting with the mobile computing device at a second particular time, the second particular time occurring a threshold amount time after the first particular time; and
responsive to determining that the user is not interacting with the mobile computing device at the second particular time, transmitting, by the mobile computing device and to a computerized watch, the notification data.
2. The method of claim 1, wherein transmitting the notification data is responsive to determining that the user is interacting with the computerized watch at the second particular time.
3. The method of claim 1, wherein transmitting the notification data is further responsive to determining that the computerized watch is being worn at the second particular time.
4. The method of claim 1, wherein transmitting the notification data is further responsive to determining that the user did not hear, feel, or view the indication of the notification data that was output by the mobile computing device at the first particular time.
5. The method of claim 1, wherein determining that the user is not interacting with the mobile computing device at the second particular time is based at least in part on determining that the user is interacting with the computerized watch at the second particular time.
6. The method of claim 1, wherein determining that the user is not interacting with the mobile computing device at the second particular time is based at least in part on determining that the mobile computing device is laying on a surface at the second particular time.
7. The method of claim 1, wherein the indication of the notification data is an initial indication, the method further comprising:
after transmitting the notification data, ceasing outputting, by the mobile computing device, the initial indication of the notification data; and
responsive to determining that the user is interacting with the mobile computing device at a third particular time, outputting, by the mobile computing device, a subsequent indication of the notification data, the third particular time occurring after the notification data is output for transmission to the computerized watch.
8. The method of claim 7, wherein the subsequent indication of the notification date is output further in response to determining that the user did not hear, feel, or view any indication of the notification data at the computerized watch.
output, at a first particular time, an indication of notification data associated with the user;
determine that the user is not interacting with the mobile computing device at a second particular time, the second particular time occurring a threshold amount time after the first particular time; and
responsive to determining that the user is not interacting with the mobile computing device at the second particular time, transmit, to a computerized watch, the notification data.
10. The mobile computing device of claim 9, wherein the at least one module is further operable by the at least one processor to transmit the notification data in response to determining that the user is interacting with the computerized watch at the second particular time.
11. The mobile computing device of claim 9, wherein the at least one module is further operable by the at least one processor to transmit the notification data in response to determining that the computerized watch is being worn at the second particular time.
12. The mobile computing device of claim 9, wherein the at least one module is further operable by the at least one processor to transmit the notification data in response to determining that the user did not hear, feel, or view the indication of the notification data that was output by the mobile computing device at the first particular time.
13. The mobile computing device of claim 9, wherein the at least one module is further operable by the at least one processor to determine that the user is not interacting with the mobile computing device at the second particular time based at least in part on determining that the mobile computing device is laying on a surface at the second particular time.
14. The mobile computing device of claim 9, wherein the indication of the notification data is an initial indication, the at least one module is further operable by the at least one processor to:
after transmitting the notification data, cease outputting the initial indication of the notification data; and
responsive to determining that the user is interacting with the mobile computing device at a third particular time, output a subsequent indication of the notification data, the third particular time occurring after the notification data is output for transmission to the computerized watch.
15. A computer-readable storage medium comprising instructions that, when executed, cause one or more processors of a mobile computing device to:
16. The computer-readable storage medium of claim 15, wherein the mobile computing device is a mobile phone or tablet computer and the wearable computing device is a computerized watch.
17. The computer-readable storage medium of claim 15, wherein the instructions, when executed, further cause the at least one processor to transmit the notification data in response to determining that the user is interacting with the computerized watch at the second particular time.
18. The computer-readable storage medium of claim 15, wherein the instructions, when executed, further cause the at least one processor to transmit the notification data in response to determining that the computerized watch is being worn at the second particular time.
19. The computer-readable storage medium of claim 15, wherein the instructions, when executed, further cause the at least one processor to transmit the notification data in response to determining that the user did not hear, feel, or view the indication of the notification data that was output by the mobile computing device at the first particular time.
20. The computer-readable storage medium of claim 15, wherein the instructions, when executed, further cause the at least one processor to determine that the user is not interacting with the mobile computing device at the second particular time based at least in part on determining that the mobile computing device is laying on a surface at the second particular time.
US15/183,142 2013-12-10 2016-06-15 Predictive forwarding of notification data Active 2034-02-10 US9853931B2 (en)
US14/102,124 US8738723B1 (en) 2013-12-10 2013-12-10 Predictive forwarding of notification data
US14/242,917 US9407591B2 (en) 2013-12-10 2014-04-02 Predictive forwarding of notification data
US15/183,142 US9853931B2 (en) 2013-12-10 2016-06-15 Predictive forwarding of notification data
US15/825,956 US20180091465A1 (en) 2013-12-10 2017-11-29 Predictive forwarding of notification data
US14/242,917 Continuation US9407591B2 (en) 2013-12-10 2014-04-02 Predictive forwarding of notification data
US15/825,956 Continuation US20180091465A1 (en) 2013-12-10 2017-11-29 Predictive forwarding of notification data
US20160301649A1 US20160301649A1 (en) 2016-10-13
US9853931B2 true US9853931B2 (en) 2017-12-26
ID=50736626
US14/102,124 Active US8738723B1 (en) 2013-12-10 2013-12-10 Predictive forwarding of notification data
US14/242,917 Active 2034-07-30 US9407591B2 (en) 2013-12-10 2014-04-02 Predictive forwarding of notification data
US15/183,142 Active 2034-02-10 US9853931B2 (en) 2013-12-10 2016-06-15 Predictive forwarding of notification data
US15/825,956 Pending US20180091465A1 (en) 2013-12-10 2017-11-29 Predictive forwarding of notification data
US (4) US8738723B1 (en)
KR (2) KR101474958B1 (en)
CN (2) CN104699646B (en)
AU (1) AU2014202038B1 (en)
DE (1) DE102014009871B4 (en)
GB (2) GB2517836B8 (en)
US20170337045A1 (en) * 2016-05-17 2017-11-23 Google Inc. Automatic graphical user interface generation from notification data
US10296762B2 (en) * 2016-07-13 2019-05-21 Fujitsu Limited Privacy enhanced push notification
US20180107930A1 (en) * 2016-10-14 2018-04-19 Google Inc. Proactive virtual assistant
CN106487928A (en) * 2016-12-09 2017-03-08 北京小米移动软件有限公司 Message pushing method and device
EP3416360A1 (en) * 2017-06-12 2018-12-19 The Swatch Group Research and Development Ltd A method of processing incoming calls
US20190012902A1 (en) * 2017-07-06 2019-01-10 Qualcomm Incorporated Intelligent alerts in multi-user environment
US20190075069A1 (en) * 2017-09-01 2019-03-07 Qualcomm Incorporated Behaviorally modelled smart notification regime
US20190158447A1 (en) * 2017-11-20 2019-05-23 Google Llc Systems, methods, and apparatus for controlling provisioning of notifications based on sources of the notifications
US20050277447A1 (en) * 2002-05-03 2005-12-15 Buil Vincentius P Mobile hand-held device
GB2420880A (en) 2004-11-26 2006-06-07 Ibm Inhibiting overlooking notifications in applications
US20070088553A1 (en) 2004-05-27 2007-04-19 Johnson Richard G Synthesized interoperable communications
US20080075066A1 (en) 2006-09-11 2008-03-27 Avaya Technology Llc Presence-Based Manager of Displayable Messages
US20090054040A1 (en) 2007-02-21 2009-02-26 Van Wijk Jacques Methods and Systems for Presence-Based Filtering of Notifications of Newly-Received Information Repository Data
US7603421B1 (en) 2004-10-25 2009-10-13 Sprint Spectrum L.P. Method and system for management of instant messaging targets
US20090276700A1 (en) 2008-04-30 2009-11-05 Nokia Corporation Method, apparatus, and computer program product for determining user status indicators
US20090307715A1 (en) 2008-06-06 2009-12-10 Justin Santamaria Managing notification service connections
US20090326791A1 (en) 2008-06-27 2009-12-31 Microsoft Corporation Local decision policies about the sharing of sensed data that enhance privacy and lower communication costs for services that aggregate data from personal devices
US20100029250A1 (en) 2008-07-29 2010-02-04 Embarq Holdings Company, Llc System and method for routing messages to communication devices associated with a user
US20110302247A1 (en) 2010-06-02 2011-12-08 Microsoft Corporation Contextual information dependent modality selection
US20120002599A1 (en) 2010-06-30 2012-01-05 Ramesh Annavajjala Implicit Channel Sounding for Closed-Loop Transmission in MIMO-OFDM Wireless Networks
US20120115451A1 (en) * 2010-11-10 2012-05-10 Cox Communications, Inc. Systems and Methods for Dynamically Forwarding Wireless Communications on Mobile Communications Devices
US20120198372A1 (en) 2011-01-31 2012-08-02 Matthew Kuhlke Communication processing based on current reading status and/or dynamic determination of a computer user's focus
US20120311096A1 (en) * 2011-06-03 2012-12-06 Apple Inc. Sending files from one device to another device over a network
CN102938787A (en) 2012-11-14 2013-02-20 王玉娇 Event source device, event showing devices and related method
US8560632B2 (en) 2007-09-20 2013-10-15 Alcatel Lucent Device for the adaptive processing of application notifications intended for communication terminals connected to a transmission infrastructure
US20130303087A1 (en) * 2012-05-08 2013-11-14 ConnecteDevice Limited Connected Device Platform
US20130316744A1 (en) * 2012-05-27 2013-11-28 Qualcomm Incorporated Notification based on user context
US20130339436A1 (en) 2012-06-19 2013-12-19 Research In Motion Limited Delayed or suspended alerts with multiple devices in proximity
WO2014014927A2 (en) 2012-07-17 2014-01-23 Google Inc. System and method for delivering alerts
US8711732B2 (en) 2004-05-27 2014-04-29 Richard G. Johnson Synthesized interoperable communications
EP2736200A1 (en) 2012-11-27 2014-05-28 Facebook, Inc. Transmission of notifications to multiple devices associated with a user
US20140244714A1 (en) 2013-02-25 2014-08-28 Google Inc. Suppression of Extraneous Alerts on Multiple Devices
US20150065055A1 (en) * 2013-08-28 2015-03-05 Qualcomm Incorporated Wirelessly Connecting Mobile Devices and Wearable Devices
US20150186092A1 (en) * 2013-12-28 2015-07-02 Mark R. Francis Wearable electronic device having heterogeneous display screens
US20160277891A1 (en) * 2013-11-04 2016-09-22 Apple Inc. Detecting stowing or unstowing of a mobile device
2013-12-10 US US14/102,124 patent/US8738723B1/en active Active
2014-04-02 US US14/242,917 patent/US9407591B2/en active Active
2014-04-10 AU AU2014202038A patent/AU2014202038B1/en active Active
2014-06-27 KR KR20140080119A patent/KR101474958B1/en active IP Right Grant
2014-07-03 DE DE102014009871.0A patent/DE102014009871B4/en active Active
2014-07-10 CN CN201410328301.7A patent/CN104699646B/en active IP Right Grant
2014-07-10 GB GB1412306.1A patent/GB2517836B8/en active Active
2014-07-10 CN CN201610245222.9A patent/CN105978785A/en active Search and Examination
2014-07-10 GB GB1501241.2A patent/GB2521286B/en active Active
2014-10-07 KR KR1020140135262A patent/KR101753270B1/en active IP Right Grant
2016-06-15 US US15/183,142 patent/US9853931B2/en active Active
2017-11-29 US US15/825,956 patent/US20180091465A1/en active Pending
CN101044470A (en) 2003-06-30 2007-09-26 微软公司 Positioning and rendering notification heralds based on user's focus of attention and activity
US8897759B2 (en) 2010-11-10 2014-11-25 Cox Communications, Inc. Systems and methods for dynamically forwarding wireless communications on mobile communications devices
US20130295886A1 (en) 2011-03-31 2013-11-07 Matthew R. Kissinger Sensor driven message retrieval for mobile device
WO2013085867A1 (en) 2011-12-08 2013-06-13 Google Inc. Contextual and location awareness for device interaction
US20140149538A1 (en) * 2012-11-27 2014-05-29 Facebook, Inc. Transmission of notifications to multiple devices associated with a user
US20150163188A1 (en) 2013-12-10 2015-06-11 Google Inc. Predictive forwarding of notification data
Allowance of Patent, and translation thereof, from counterpart Korean application No. 10-2014-0135262, dated May 30, 2017, 3 pp.
Combined Search and Examination Report from GB Application 1412306.1, dated Jul. 31, 2014, 7 pp.
Examination Report from counterpart Australian Patent Application No. 2014206208, dated Dec. 10, 2015, 2 pp.
Notice of Acceptance from Australian Application No. 2014206208, dated Jan. 7, 2016 7 pgs.
Notice of Allowance from U.S. Appl. No. 14/102,124, dated Feb. 28, 2014, 8 pgs.
Notice of Office Action, and translation thereof, from counterpart Korean Application No. 10-2014-0135262, dated Jan. 18, 2017 5 pgs.
Notification of First Office Action from Chinese Patent Application No. 201410328301.7, dated Oct. 29, 2015 10 pgs.
Prosecution History from U.S. Appl. No. 14/242,917, from Apr. 3, 2014 through Apr. 4, 2016, 27 pgs.
Response to Combined Search and Examination Report from British application No. 1501241.2, filed Jun. 19, 2015 15 pp.
Response to Combined Search and Examination Report from GB Application No. 1412306.1, filed Dec. 1, 2014 13 pp.
Search and Examination Report from counterpart British Patent Application No. 1501241.2, dated Feb. 24, 2015, 7 pp.
US20150163188A1 (en) 2015-06-11
GB201501241D0 (en) 2015-03-11
GB201412306D0 (en) 2014-08-27
US8738723B1 (en) 2014-05-27
GB2517836A8 (en) 2015-10-21
GB2517836A (en) 2015-03-04
DE102014009871B4 (en) 2018-08-23
KR20150067712A (en) 2015-06-18
DE102014009871A1 (en) 2015-06-11
KR101753270B1 (en) 2017-07-03
US20160301649A1 (en) 2016-10-13
CN104699646A (en) 2015-06-10
CN104699646B (en) 2016-04-13
CN105978785A (en) 2016-09-28
GB2521286A (en) 2015-06-17
US9407591B2 (en) 2016-08-02
GB2517836B (en) 2015-08-05
KR101474958B1 (en) 2014-12-19
US20180091465A1 (en) 2018-03-29
GB2517836B8 (en) 2015-10-21
AU2014202038B1 (en) 2014-07-03
GB2521286B (en) 2015-10-21
EP3447624A1 (en) 2019-02-27 Systems and methods for proactively identifying and surfacing relevant content on a touch-sensitive device
WO2012019350A1 (en) 2012-02-16 Finger identification on a touchscreen
CN104838353A (en) 2015-08-12 Coordination contextual display data on a display screen
US8990344B2 (en) 2015-03-24 Custom event and attraction suggestions
DE102014009871A1 (en) 2015-06-11 Previously meaningless right forward message data
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:FAABORG, ALEXANDER;ROBISON, AUSTIN;SIGNING DATES FROM 20131119 TO 20131203;REEL/FRAME:038920/0418