Abstract:
Inventive embodiments described herein relate to a universal approach for handling user interface notifications. A notification pipeline on a device acts as an intermediary between applications on the device and notification surfaces on the device. Notification surfaces register themselves with the notification pipeline to become active subscribers. Applications or other processes submit notifications complying with a same format, schema, structure, etc. to the notification pipeline. The notification pipeline broadcasts the notification to the subscribed notification surfaces, which in turn determine whether or how to handle and perhaps display the notification based on the content of the notification.

Description:
RELATED APPLICATIONS 
       [0001]    This application is related to U.S. Provisional Application No. 62/316,526 (attorney docket number 359578.01), titled “UNIVERSAL NOTIFICATION PIPELINE”, filed Mar. 31, 2016, and incorporated herein by reference. 
     
    
     BACKGROUND 
       [0002]    A goal of computing is to provide information to people. Assuming that a person uses a computing device to access information, competing factors can bear on how and when information should be presented. For example, a user&#39;s goals may vary according to how much time is available, what activity the user is engaged in, the importance of available information, etc. The amount and type of information that is available and the frequency at which it becomes available can vary significantly. Moreover, computing devices that display information have a wide range of power capabilities, computing capacity, display size, and types of input devices. A given piece of information could potentially be destined for a small device worn on a wrist and a device with a large display in a building lobby. 
         [0003]    To accommodate the variety and complexity of information presentation factors, software developers have designed different types of notification mechanisms or notification surfaces. For example, many operating systems or graphical user shells provide different notification surfaces for different contexts. For instance, a lock screen or login screen might have a notification surface, a “home” or “start” screen for launching applications might have icons capable of displaying notifications or information about them, a tool or status bar might have a user interface element that can be activated to display notifications, or a “toast” notification surface might be provided. In any case, these kinds of notification surfaces often have different sizes, consume different types or formats of information, implement different semantics, and are interfaced through different application programming interface (API) or protocols. 
         [0004]    The inventors alone have realized that it is becoming difficult for developers to write software that can take advantage of the variety of notification surfaces available on different platforms. The inventors have appreciated that a developer writing software to disseminate a single item of information might currently need to write many sections of code, each customized for respective notification surfaces. In addition, if a single item of notification information is intended to be displayed by multiple notification surfaces of a same device, multiple notification messages of varying content and format might need to be transmitted, resulting in duplicative transmission of the same item of information, possibly resulting in multiple process “wake ups” and other overhead at the receiving device. Consequently, the inventors alone have realized that providing a universal notification mechanism on a computing device can potentially ease some of the problems related to the proliferation of notification surfaces. 
       SUMMARY 
       [0005]    The following summary is included only to introduce some concepts discussed in the Detailed Description below. This summary is not comprehensive and is not intended to delineate the scope of the claimed subject matter, which is set forth by the claims presented at the end. 
         [0006]    Inventive embodiments described herein relate to a universal approach for handling user interface notifications. A notification pipeline on a device acts as an intermediary between applications on the device and notification surfaces on the device. Notification surfaces may register themselves with the notification pipeline to become active subscribers to the pipeline. Applications or other processes submit to the notification pipeline notifications complying with a same format, schema, structure, etc. The notification pipeline broadcasts the notification to the subscribed notification surfaces, which in turn determine whether or how to handle and perhaps display information in the notification based on the content of the notification. 
         [0007]    Many of the attendant features will be explained below with reference to the following detailed description considered in connection with the accompanying drawings. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0008]    The present description will be better understood from the following detailed description read in light of the accompanying drawings, wherein like reference numerals are used to designate like parts in the accompanying description. 
           [0009]      FIG. 1  shows examples of notification surface graphics. 
           [0010]      FIG. 2  shows a computing device and software environment within which notification surfaces may execute. 
           [0011]      FIG. 3  shows an example of a computing device with disparate notification surfaces. 
           [0012]      FIG. 4  shows details of an example scenario of uncoordinated use of notification surfaces. 
           [0013]      FIG. 5  shows an embodiment of a notification pipeline. 
           [0014]      FIG. 6  shows another embodiment of the notification pipeline. 
           [0015]      FIG. 7  shows an example of how a notification subscription list can change over time. 
           [0016]      FIG. 8  shows how different notification surfaces might each handle a same notification. 
           [0017]      FIG. 9  conceptually shows how the notification pipeline enables different notification publishers to specify categorical notification intentions to subscribers. 
           [0018]      FIG. 10  shows an example of a computing device. 
       
    
    
     DETAILED DESCRIPTION 
       [0019]    Embodiments discussed below relate to a universal notification pipeline. Notification surfaces in general will be described first, including some graphical attributes, their technical features, and how they have been implemented. A notification pipeline is described next. A publisher-subscriber design is explained, as well as techniques to integrate the notification pipeline into existing notification surfaces or managers. 
         [0020]      FIG. 1  shows examples of notification surface graphics that might be displayed by a same computing device. As discussed in the Background, a variety of notification surfaces are known and in use. Embodiments described herein can be conveniently used by existing and new notification surfaces. The term “notification surface” as used herein refers to pieces of executable software on a computing device and the graphical user interface (GUI) elements that they generate and control. Generally, most notification surfaces have been accessible, through respective unique APIs or the like, to any software or application executing on the relevant computing device. Most notification surfaces have distinct behavior and appearance. As shown in  FIG. 1 , notification surfaces are often incorporated in graphical user shells in the form of dynamic icons  100 , information panels  102  as might appear in a “desktop” or side bar of a graphical shell, a toast widget  104 , etc. Some notification surfaces, as part of a graphical user shell, are able to “pop up” or overlay other graphics. Some notifications may be incorporated into a login or lock screen. Some notification surfaces may have interactive GUI elements  108  for dismissing notifications, opening related content or applications, and so forth. Some notification surfaces may only display information non-interactively. Some notification surfaces are autonomous operating system processes, whereas some are integrated into an operating system. 
         [0021]      FIG. 2  shows a computing device  120  and software environment within which notification surfaces may execute. The computing device  120  has an operating system  122  managing processes  124  including a windowing system  125  that, for some of the processes (e.g., applications) manages the display of windows  126  displayed on a display  128 . The computing device  120  may be any of a variety of types which are described later with reference to  FIG. 10 . The computing device  120  has at least storage hardware and processor hardware for executing the operating system  122  and the processes  124 . The windowing system  125  may be any type of windowing system, for instance tiled, “free form”, single window (only one application window is displayed at a time), etc. Processes are well known operating system objects that are discretely managed units of execution, typically having a process identifier, memory managed by the operating system  122  (heap and/or stack memory), and may be managed in a process table of process identifiers or the like that is maintained by the operating system  122 . 
         [0022]      FIG. 3  shows an example of a computing device with disparate notification surfaces. As noted, it is common for a GUI shell  140  or environment to include different types of notification surfaces. The notification surfaces may execute as respective processes. For instance, a toast manager  142 , badge manager  144 , login screen  146 , and statusbar  148  (or sidebar, toolbar, etc.) may be respective processes or distinction components/interfaces of a same process. Some GUI shells also include a tile manager (not shown) that manages the appearance, display, layout, and user-activations of dynamic tiles or icons, which function as both application-launching icons as well as mini-windows that can display notifications for represented applications. 
         [0023]    Local software elements, typically applications  150 , execute on the computing device  120  and submit notifications to whichever notification surfaces they are programmed to interface with. To accommodate remote notifications received from a network via a network interface card  152  and network stack  154 , a push listener  156  background process may receive and handle network notifications, for instance in the form of HyperText Transfer Protocol (HTTP) messages. As can be seen in  FIG. 3 , the each notification client has had to communicate directly with whichever notification surfaces it intends to use. 
         [0024]      FIG. 4  shows details of an example scenario of uncoordinated use of notification surfaces. Each of the depicted notification surfaces have their own APIs  142 A,  144 A,  148 A. The shades of the notifications in  FIG. 4  correspond to the shades of their respective APIs  142 A,  144 A,  148 A. A push notification service  160  might receive a message from a user device  162  (e.g., a smartphone), which push the notification service  160  uses to build a notification N1 that is pushed via a network  164 , for instance as an HTTP message containing an application-identifying unique Uniform Resource Identifier (URI), to the push listener  156 . The push notification service  160  also receives a notification N2 relayed by a cloud service  166  from another user device  162 . The notification N2 is received by the push listener  156 , which passes N2 to application  150  (app1), which might translate N2 into a corresponding data structure or object that is passed via API  142 A (API1) to the toast manager  142 A. The cloud service  166  also originates its own notifications N3 and N4. Notification N3 is transmitted directly to a custom network listener of the badge manager  144 , and notification N4 is transmitted to application  152  (app2) which, possibly after reformatting or repackaging the N4 notification, invokes the corresponding API  148 A (API3) to submit the notification to the statusbar  148 . The application  150  (app2) also originates a notification N5 for the API  144 A (API2) of the badge manager  144 . Similarly, application  150  (app1) originates notification N6 that is submitted to the badge manager  144 . The relevant notification surfaces receive and handle their notifications in known ways, which will usually involves rendering a notification graphic to inform a person using the computing device  120 . 
         [0025]    As can be seen in  FIG. 4 , each client requires different code for each notification surface it intends to use. Even if a common format or schema is used by all of the notification surfaces for encapsulating notification content, different APIs are involved and separate communications are needed for each notification surface that is to receive a same piece of notification content or information. In addition, in the case of stateful notifications, if a client intends to use different notification surfaces to give notices of a same piece of information, if the state of one notification changes, ideally, the client must detect the notification change and communicate with the other notification surfaces to inform them of the status change. 
         [0026]      FIG. 5  shows an embodiment of a notification pipeline  180 . Depending on how it is implemented, the notification pipeline  180  may obviate some of the shortcomings of dealing with multiple notification surfaces without significant modification of existing notification surfaces or systems. The notification pipeline  180  implements a publish-subscribe (“pub-sub”) design pattern. Notification surfaces and clients thereof use the notification pipeline  180  to intermediate exchanges of notifications. As shown in  FIG. 5 , the clients such as push listener  156 A and applications  150 A differ from their counterparts discussed earlier. The notification surfaces  142 B,  144 B,  146 B, and  148 B also differ from their preceding siblings. Both the clients and the notification surfaces are modified in several ways. 
         [0027]    First, clients and notification surfaces should be built or modified to use a common notification format. An aspect of the pub-sub design of the notification pipeline  180  is that each notification received from a client by the notification pipeline  180  is passed to each notification surface that is currently subscribed to the notification pipeline  180 . Any client or notification surface that participates should implement a same notification format or data structure. For example, if notification payloads are implemented as eXtensible Mark-up Language (XML) code, then each notification should conform to a same XML schema. In that case, clients should be programmed to format their notifications as XML code conformant to a notification schema. Examples are described in the application referenced above in the “RELATED APPLICATIONS” section. Other types of data packaging or formatting may be used for notifications. For instance, well-defined data structures (e.g., JavaScript Object Notation), portable objects, sets of key-value pairs in text form, etc. Of note will be a “contract” or understanding that any subscribed notification surface will be able to at least parse and interpret (possibly using other facilities) a notification that complies with the agreed-upon data format. How a notification surface handles the content of a compliant notification, if at all, is up to the notification surface. Although, semantic conventions may be attached to features of notifications. 
         [0028]    Second, the clients and notification surfaces should be coded to use a same API or other type of software interface to access the notification pipeline  180 . A notification pipeline API can be as simple as a “subscribe( )” call, a “publish( )” call, and possibly an “unsubscribe( )” call, although other calls or methods may be helpful, as discussed later. As described in detail below, notification surfaces call “subscribe( )” or an equivalent to become subscribers and consequently receive notifications. Clients call “publish( )” to pass a notification as a parameter, event, or the like. 
         [0029]    The left side of  FIG. 5  shows subscribing steps. At step  182  the subscribe( ) function is invoked, and the notification pipeline  180  responds at step  184  by adding the caller to a list of current subscribed notification surfaces (see the subscription list  204  in  FIG. 6 ). The left side of  FIG. 5  also shows publishing steps. At step  186  a client such as push listener  156 A calls the subscribe( ) function to submit a standardized notification (one conforming to the “contract”). In response, at step  188 , the notification pipeline  180  publishes the notification to all of the current notification surfaces that have registered with the subscribe( ) call. At step  190 , the subscribers receive the notification. 
         [0030]      FIG. 6  shows an embodiment of the notification pipeline  180 . The same shading of notifications N1-N5 and the API  200  reflect the use of a common notification scheme or format and the same API  200  (or other type of software interface) by the clients and notification surfaces. In  FIG. 6 , it is assumed that each of the exemplary notification surfaces have already invoked the subscribe function  202  through the API  200  and that the notification pipeline  180  has added respective identifiers  203  to a subscription list  204 . When the push listener  156 A is to issue a notification N1, the push listener  156 A invokes the publish function  206 . In response, the publish function  206  is executed for notification N1. The publish function obtains the identifiers from the subscription list  204  and passes the notification N1 to each of the identified notification surfaces. Consequently, with one API call and one notification message or payload, the push listener  156 A, for instance, is able to provide same notification content to multiple notification surfaces. Similarly, notifications N2-N5 are published. Each time, one notification transaction from a publishing client results in a broadcast of a notification. 
         [0031]    In one embodiment, an inter-process communication (IPC) mechanism  208  may be used to pass notifications from the notification pipeline  180  to the subscribed notification surfaces. For instance, an event routing/sinking mechanism of the operating system may be used, where each notification surface listens for a notification-type event and implements an event handler for the same. Other IPC mechanisms may be used, for instance shared memory, message queues, pipes, etc. Although overhead and performance might be prohibitive, the notification pipeline can also be implemented as an application-level protocol built on top of the local network protocol stack. Notifications could be formed as HTTP requests, for instance. Internally generated notifications would only pass through a loopback interface and notifications from a network would be directly handled by the pipeline (a push listener, for instance, might be obviated). An IPC approach will likely be preferable. 
         [0032]    Regarding the assumption that various notification surfaces have already subscribed, it is convenient for a notification surface to invoke the subscribe function  202  in its initialization code. A typical notification surface will begin executing at boot time, login, etc. Some notification surfaces may take advantage of the unsubscribe function  210  (see  FIG. 7 ) to reduce computation overhead, reduce power consumption, or accommodate some application-specific logic. 
         [0033]      FIG. 7  shows an example of how the subscription list  204  can change over time. First, to handle a notification publication request, the publish function  206  reads the subscriber table or list  204  and finds subscriber identifiers  212  ID1 and ID2, to which the notification request is sent. Next, a notification surface invokes the unsubscribe function  210  to remove itself from the subscriber list  204 . Although ID2 is shown as a parameter, in practice, a notification surface might not need to explicitly identify itself; the notification pipeline  180  accesses the invoker&#39;s execution context or the like to identify the subscribing or unsubscribing surface. Next, a third notification surface invokes the subscribe function  202  which in turn adds its identifier (ID3) to the subscriber list  204 . Responsive to a second invocation of the publish function  206  to publish another notification, the subscriber list  204  is again read and the current subscriber identifiers  212 —ID1 and ID3—are used to send the second notification. 
         [0034]      FIG. 6  also shows an action function  214 . The action function  214  can be any type of action that can be directed to notifications. For example, the action function  214  can be a “dismiss( )” function invocable by any notification surface, client, or other software. The dismiss function corresponds to a dismissal of a notification. If a notification surface invokes the dismiss function, the call might reflect a user interaction with the corresponding surface to dismiss a notification. The notification pipeline  180  identifies the surface that called the dismiss function, and publishes a dismiss message or event to the other subscribed notification surfaces. Each notification surface that receives the dismissal message may respond by dismissing the identified notification. Other types of actions or control logic may be provided to coordinate actions of the notification surfaces, thus reducing the notification-related logic needed by individual applications (clients). 
         [0035]    The notification pipeline  180  may also implement gatekeeper functionality. For instance, the notification pipeline  180  can check each incoming notification for conformance with the standard schema or format, which can alleviate the notification surfaces from each having to perform the same function. 
         [0036]      FIG. 8  shows how different notification surfaces  230  might each handle a same notification  232 . The notification surface  230  depicted on the left might receive the notification  232 , parse out the elements, and evaluate which elements are present and what their attributes and inner values are. From that information, the logic of the notification surface determines to render the notification  232 . The notification surface  230  depicted in the center similarly receives and parses the same notification  232 . However, the logic for this surface determines that the notification should be ignored. The notification surface  230  depicted on the right decided that the notification  232  is applicable, but rather than render the notification  232 , the notification surface forwards the notification  232 , for example, to an agent or proxy of another device (e.g., a wearable device). 
         [0037]      FIG. 9  conceptually shows how the notification pipeline enables different notification publishers  240  to specify categorical notification intentions to subscribers  242 . As noted in the BACKGROUND section, many variables can affect notification choices. When an application is coded and compiled, it is unlikely that a developer will know exactly what kinds of devices the application will run on, which notification surfaces will be available or what their characteristics such as size and capabilities might be. If a universal notification schema is used, as in the above-referenced related patent application, a notification publisher  240  can define bindings for which a notification should be applied. 
         [0038]    For instance, a notification might be defined as being bound to “medium” sized notification surfaces or notification surfaces greater than a certain size. Different style hints or notification content might be associated with different bindings. Put another way, a notification publisher need not specify particular notification surfaces, but rather specifies the types or properties for which a notification is intended, all subscribed notification surfaces receive the notification, and each subscriber  242  handles the notification according to its own traits compare to the specifications of the notification. In  FIG. 9 , if publisher1 has published a notification with few if any requirements, then each subscriber acts on the notification. If publisher2 publishes a notification that specifies a large notification surface, and only subscriber2 and subscriber3 meet the requirement, then they display the notification and the other subscribers ignore it. If publisher3 submits to the pipeline a notification that includes text with a specification that the text is only to be rendered as audio, and if only subscriber4 has such a capability, then only subscriber4 supplies the information to the user; the other subscribers take no action. By the same logic, publishers can specify intentions for how different portions of content or semantics of a notification are to be handled. 
         [0039]    The notification pipeline can also help avoid forward and backward compatibility issues. Further to that purpose, the standard format or schema can include an “optional” construct or tag. Each surface is guaranteed to be able to at least recognize and parse the optional tag. Older surfaces might ignore such optional notification content, but newer surfaces might be programmed to act upon the optional content. As new notification surface features arise, newer clients will be able to plug into them without affecting backward compatibility of the older clients. For example, if a software developer wants to support video in a toast, and perhaps a first version of a graphical shell does not support video toast but a second version does, if the notifying client puts the video content in the optional part of the notification, then the same notification payload can be sent without regard for whether the first or second version will be handling it. 
         [0040]    In some cases, a notification surface might need to know which application on its computing device submitted a notification. As noted above, the notification pipeline can automatically identify which application has submitted a notification and provide that information with the broadcasted publication of the notification. This allows the application-specific recipient to direct the content of the notification to the appropriate sub-surface or object. For instance, if a badge style notification is published with the identity of the application that published it, a recipient can update the icon of the corresponding application. In other words, the notification pipeline can be designed to enable implicit association of applications and notifications. Two different processes with multiple notification surfaces, all or many of which are associated with particular respective applications, can, through one subscription, receive notifications from the respective applications and direct them to the appropriate application-specific notification surfaces. 
         [0041]      FIG. 10  shows details of a computing device  120  on which embodiments described above may be implemented. The technical disclosures herein constitute sufficient information for programmers to write software, and/or configure reconfigurable processing hardware (e.g., FPGAs), and/or design application-specific integrated circuits (ASICs), etc., to run on one or more of the computing devices  120  to implement any of features or embodiments described in the technical disclosures herein. 
         [0042]    The computing device  120  may have a display  252 , a network interface  254 , as well as storage hardware  256  and processing hardware  258 , which may be a combination of any one or more: central processing units, graphics processing units, analog-to-digital converters, bus chips, FPGAs, ASICs, Application-specific Standard Products (ASSPs), or Complex Programmable Logic Devices (CPLDs), etc. The storage hardware  256  may be any combination of magnetic storage, static memory, volatile memory, non-volatile memory, optically or magnetically readable matter, etc. The meaning of the term “storage”, as used herein does not refer to signals or energy per se, but rather refers to physical apparatuses and states of matter. The hardware elements of the computing device  250  may cooperate in ways well understood in the art of computing. In addition, input devices may be integrated with or in communication with the computing device  250 . The computing device  250  may have any form factor or may be used in any type of encompassing device. The computing device  250  may be in the form of a handheld device such as a smartphone, a tablet computer, a gaming device, a server, a rack-mounted or backplaned computer-on-a-board, a system-on-a-chip, or others. 
         [0043]    Embodiments and features discussed above can be realized in the form of information stored in volatile or non-volatile computer or device readable storage hardware. This is deemed to include at least storage hardware such as optical storage (e.g., compact-disk read-only memory (CD-ROM)), magnetic storage hardware, flash read-only memory (ROM), and the like. The information stored in storage hardware can be in the form of machine executable instructions (e.g., compiled executable binary code), source code, bytecode, or any other physical hardware having a physical state that can transfer information to processing hardware to enable or configure computing devices to perform the various embodiments discussed above. This is also deemed to include at least volatile memory such as random-access memory (RAM) and/or virtual memory storing information such as central processing unit (CPU) instructions during execution of a program carrying out an embodiment, as well as non-volatile media storing information that allows a program or executable to be loaded and executed. The embodiments and features can be performed on any type of computing device, including portable devices, workstations, servers, mobile wireless devices, and so on.