Patent Publication Number: US-9892105-B2

Title: Systems and methods for annotating messages with reminders

Description:
TECHNICAL FIELD 
     The disclosure relates generally to electronic messaging applications, and more specifically to annotating messages with task reminders. 
     BACKGROUND 
     Electronic messages, such as email, convey various types of information to the recipient. In some instances, a message is purely informational, and may be filed away or deleted after it is read. Some messages result in one or more tasks that the user must perform. Some people leave messages in an inbox pending completion of the associated tasks, or enter associated tasks in a personal calendar or task management system. Managing the tasks associated with messages can be cumbersome or inconvenient. 
     SUMMARY 
     Disclosed implementations address the above deficiencies and other problems associated with managing tasks associated with electronic messages. To avoid the burden of maintaining a separate task management system and entering tasks into that system, disclosed implementations enable a user to annotate messages with task reminders. Further, when an electronic message is annotated with one or more tasks, the descriptions of the tasks are displayed in the message list rather than displaying a summary of the electronic message. In this way, a user can easily see which messages have corresponding tasks and what those tasks are. 
     In accordance with some implementations, a method of annotating an electronic message executes at a computing device having one or more processors and memory. The memory stores one or more programs configured for execution by the one or more processors. The method displays a user interface for an electronic messaging application. In some implementations, the electronic messaging application is an email application. The user interface includes a list of messages received by the user, and each displayed message includes a respective message summary that is displayed. The method receives from the user a description of a first task associated with a first message of the displayed messages. The user provides the description using a control in the user interface. In some implementations, the user can provide the description by selecting from a list of task description options displayed by the control. The method stores the description as a first annotation associated with the first message and updates the displayed messages in the list of messages. The method displays for the first message the first annotation as a substitute for the message summary. 
     In some instances, the method subsequently receives input from the user to dismiss the first task (e.g., by designating the first task as completed). In this case, the method updates the displayed messages in the list of messages, displaying for the first message the message summary instead of the first annotation. 
     In some implementations, the user interface includes a task view control that toggles whether message summaries or task annotations are displayed in the list. When the task view control is in a first state, the method displays task descriptions instead of message summaries for each displayed message that has a user-annotated task description, and displays message summaries for each displayed message that has no user-annotated task description. When the task view control is in a second state, the method displays message summaries for all of the displayed messages, regardless of whether any of the displayed messages have user-annotated task descriptions. 
     In some instances, the user closes the electronic messaging application, and subsequently reopens the electronic messaging application. When the electronic messaging application is reopened, the annotations are not lost. When the method redisplays the list of messages, the first message is displayed with the first annotation as a substitute for the message summary. 
     In some implementations, a task description is intended to be a short description of all work that a user needs to perform with respect to the message. Even if the task could be subdivided into smaller tasks, there is only one message, and thus a single task description associated with that message. 
     In some implementations, the user interface allows the user to define more granular tasks. For example, some implementations allow a user to provide one or more descriptions of additional tasks associated with the first message. Each of these additional task descriptions is stored as an additional annotation associated with the first message. The method updates the displayed messages in the list of messages, displaying for the first message the first annotation and the additional annotations as a substitute for the message summary. In some implementations, each of the first annotation and additional annotations is displayed on a separate row in the list of messages, with the respective annotation instead of the message summary of the first message. In other implementations, all or some of the task descriptions are displayed on a single row in the list of messages. 
     In implementations that support multiple tasks associated with a single message, a user may designate one or more of the first task and additional tasks as completed. When one or more of the first task and additional tasks remain uncompleted, the method displays for the first message the annotations corresponding to the uncompleted tasks. The annotations corresponding to the uncompleted tasks are displayed instead of the message summary of the first message. When the first task and the additional tasks are all completed, the method displays for the first message the message summary of the first message. 
     Thus methods and systems are provided that enable a user to annotate electronic messages with task descriptions and use the annotations as a task list. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       For a better understanding of the aforementioned implementations of the invention as well as additional implementations thereof, reference should be made to the Description of Implementations below, in conjunction with the following drawings in which like reference numerals refer to corresponding parts throughout the figures. 
         FIG. 1  illustrates a context in which some implementations operate. 
         FIG. 2  is a block diagram of a client computing device according to some implementations. 
         FIG. 3  is a block diagram of a server according to some implementations. 
         FIGS. 4A-4G  illustrate aspects of a user interface for annotating electronic messages in accordance with some implementations. 
         FIGS. 5A-5C  provide a flowchart of a process, performed at a client computing device, for annotating electronic messages according to some implementations. 
     
    
    
     Reference will now be made in detail to implementations, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that the present invention may be practiced without these specific details. 
     DESCRIPTION OF IMPLEMENTATIONS 
       FIG. 1  is a block diagram that illustrates the major components of some implementations. The various client devices  102  (also identified herein as computing devices) and servers  300  in a server system  110  communicate over one or more networks  108  (such as the Internet). A client device  102  may be a smart phone, a tablet computer, a laptop computer, a desktop computer, or other computing device that has access to a communication network  108  and can run a messaging application  106 . In some implementations, the messaging application runs within a web browser  104 . 
     In some implementations, the server system  110  consists of a single server  300 . More commonly, the server system  110  includes a plurality of servers  300 . In some implementations, the servers  300  are connected by an internal communication network or bus  122 . The server system  110  includes one or more web servers  112 , which receive requests from users (e.g., from client devices  102 ) and return appropriate information, resources, links, and so on. In some implementations, the server system  110  includes one or more application servers  114 , which provide various applications, such as a messaging application  106 . The server system  110  typically includes one or more databases  116 , which store information such as web pages, a user list  118 , and various user information  120  (e.g., user names and encrypted passwords, user preferences, and so on). 
       FIG. 2  is a block diagram illustrating a client device  102  that a user uses to access a messaging application  106 . A client device is also referred to as a computing device, which may be a tablet computer, a laptop computer, a smart phone, a desktop computer, a PDA, or other computing device than can run a messaging application  106  and has access to a communication network  108 . A client device  102  typically includes one or more processing units (CPUs)  202  for executing modules, programs, or instructions stored in memory  214  and thereby performing processing operations; one or more network or other communications interfaces  204 ; memory  214 ; and one or more communication buses  212  for interconnecting these components. The communication buses  212  may include circuitry (sometimes called a chipset) that interconnects and controls communications between system components. A client device  102  includes a user interface  206  comprising a display device  208  and one or more input devices or mechanisms  210 . In some implementations, the input device/mechanism includes a keyboard and a mouse; in some implementations, the input device/mechanism includes a “soft” keyboard, which is displayed as needed on the display device  208 , enabling a user to “press keys” that appear on the display  208 . 
     In some implementations, the memory  214  includes high-speed random access memory, such as DRAM, SRAM, DDR RAM or other random access solid state memory devices. In some implementations, memory  214  includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid state storage devices. In some implementations, the memory  214  includes one or more storage devices remotely located from the CPU(s)  202 . The memory  214 , or alternately the non-volatile memory device(s) within memory  214 , comprises a non-transitory computer readable storage medium. In some implementations, the memory  214 , or the computer readable storage medium of memory  214 , stores the following programs, modules, and data structures, or a subset thereof:
         an operating system  216 , which includes procedures for handling various basic system services and for performing hardware dependent tasks;   a communications module  218 , which is used for connecting the client device  102  to other computers and devices via the one or more communication network interfaces  204  (wired or wireless) and one or more communication networks  108 , such as the Internet, other wide area networks, local area networks, metropolitan area networks, and so on;   a display module  220 , which receives input from the one or more input devices  210 , and generates user interface elements for display on the display device  208 ;   a web browser  104 , which enables a user to communicate over a network  108  (such as the Internet) with remote computers or devices;   a messaging application  106 , which enables the user to send and receive electronic messages. In some implementations, the messaging application is an email application. In some implementations, the messaging application is an instant messaging application. In some implementations, the messaging application  106  runs within the web browser  104  as illustrated in  FIG. 1 . In some implementations, the messaging application  106  runs independently of a web browser  104  (e.g., a desktop application). An example messaging application is illustrated below in  FIGS. 4A-4E ; and   application data  222 , which is used by the messaging application  106 . The application data includes messages  224  (e.g., email messages or instant messages), as well as annotations  226  for some of the messages. The application data  222  may include configuration data  228 , such as user preferences, user history, or the state of configuration options.       

     Each of the above identified executable modules, applications, or sets of procedures may be stored in one or more of the previously mentioned memory devices and corresponds to a set of instructions for performing a function described above. The above identified modules or programs (i.e., sets of instructions) need not be implemented as separate software programs, procedures, or modules, and thus various subsets of these modules may be combined or otherwise re-arranged in various implementations. In some implementations, the memory  214  may store a subset of the modules and data structures identified above. Furthermore, the memory  214  may store additional modules or data structures not described above. 
     Although  FIG. 2  shows a client device  102 ,  FIG. 2  is intended more as a functional description of the various features that may be present rather than as a structural schematic of the implementations described herein. In practice, and as recognized by those of ordinary skill in the art, items shown separately could be combined and some items could be separated. 
       FIG. 3  is a block diagram illustrating a server  300  that may be used in a server system  110 . A typical server system includes many individual servers  300 , which may be hundreds or thousands. A server  300  typically includes one or more processing units (CPUs)  302  for executing modules, programs, or instructions stored in the memory  314  and thereby performing processing operations; one or more network or other communications interfaces  304 ; memory  314 ; and one or more communication buses  312  for interconnecting these components. The communication buses  312  may include circuitry (sometimes called a chipset) that interconnects and controls communications between system components. In some implementations, a server  300  includes a user interface  306 , which may include a display device  308  and one or more input devices  310 , such as a keyboard and a mouse. 
     In some implementations, the memory  314  includes high-speed random access memory, such as DRAM, SRAM, DDR RAM or other random access solid state memory devices. In some implementations, the memory  314  includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid state storage devices. In some implementations, the memory  314  includes one or more storage devices remotely located from the CPU(s)  302 . The memory  314 , or alternately the non-volatile memory device(s) within memory  314 , comprises a non-transitory computer readable storage medium. In some implementations, the memory  314 , or the computer readable storage medium of memory  314 , stores the following programs, modules, and data structures, or a subset thereof:
         an operating system  316 , which includes procedures for handling various basic system services and for performing hardware dependent tasks;   a communications module  318 , which is used for connecting the server  300  to other computers via the one or more communication network interfaces  304  (wired or wireless), an internal network or bus  122 , or other communication networks  108 , such as the Internet, other wide area networks, local area networks, metropolitan area networks, and so on;   a display module  320 , which receives input from one or more input devices  310 , and generates user interface elements for display on a display device  308 ;   one or more web servers  112 , which receive requests from the client device  102 , and return responsive web pages, resources, or links. In some implementations, each request is logged in the database  116 ;   one or more application servers  114 , which provide various applications (such as an email or other messaging application  106 ) to the client devices  102 . In some instances, applications are provided as a set of web pages, which are delivered to the client devices  102  and displayed in a web browser  104 . The web pages are delivered as needed or requested. In some instances, an application is delivered to a client device  102  as a download, which is installed and run from the client device  102  outside of a web browser  104 ;   one or more databases  116 , which store various data used by the modules or programs identified above. In some implementations, the database  116  includes a list of authorized users, which may include user names, encrypted passwords, and other relevant information about each user. The database  116  also stores user specific data  120  that is used by one or more of the applications provided by the application server. For example, some implementations store the electronic messages  224  for each user. When a user has annotated an electronic message, the annotations  226  are also stored in the database  116 .       

     Each of the above identified elements in  FIG. 3  may be stored in one or more of the previously mentioned memory devices. Each executable program, module, or procedure corresponds to a set of instructions for performing a function described above. The above identified modules or programs (i.e., sets of instructions) need not be implemented as separate software programs, procedures or modules, and thus various subsets of these modules may be combined or otherwise re-arranged in various implementations. In some implementations, the memory  314  may store a subset of the modules and data structures identified above. Furthermore, the memory  314  may store additional modules or data structures not described above. 
     Although  FIG. 3  illustrates a server  300 ,  FIG. 3  is intended more as functional illustration of the various features that may be present in a set of one or more servers rather than as a structural schematic of the implementations described herein. In practice, and as recognized by those of ordinary skill in the art, items shown separately could be combined and some items could be separated. The actual number of servers used to implement these features, and how features are allocated among them, will vary from one implementation to another, and may depend in part on the amount of data traffic that the system must handle during peak usage periods as well as during average usage periods. 
     As illustrated in  FIGS. 2 and 3 , the functionality for a messaging application may be shared between a client device  102  and a server system  110 . In some implementations, after the messaging application is installed on a client device  102 , the majority of the subsequent processing occurs on the client device. For example, annotations may be stored solely at the client device, and not on a server  300 . In other implementations, the majority of the processing and data storage occurs at a server  300 , and the client device  102  uses a web browser  104  to view and interact with the data. For example, the annotations may be stored only at the server system  110 . One of skill in the art recognizes that various allocations of functionality between the client device  102  and the server system  110  are possible, and some implementations support multiple configurations (e.g., based on user selection). 
       FIGS. 4A-4G  illustrate features of a messaging application that supports annotations in accordance with some implementations.  FIG. 4A  illustrates an electronic message list  402 , which may be displayed as part of an electronic messaging application (e.g., an email application). In this example, each row in the list represents a separate message (or a conversation or cluster, including two or more messages). The tag column  404  includes boxes that may be used to tag messages that will be acted on jointly. For example, a user may tag multiple messages that will be deleted or moved to a same folder. The star column  406  enables a user to identify important messages. Some implementations allow a user to manually highlight a star in a row to indicate the importance of the row, as illustrated in the fifth row  428 . In some implementations, a star is highlighted based on the sender&#39;s designation of the message as important. In some implementations, a user may specify rules for which messages should be automatically tagged as important. In some implementations, when a message is starred, the message summary  416  is always displayed in the message list  402 , even when the message has an associated task. 
     Some implementations include an attachment column  408 , which identifies which messages include one or more attachments (e.g., the first message  420  and the fifth message  428  in the list  402 ). In some implementations, a paperclip icon is used to indicate the presence of an attachment. 
     In some implementations, the displayed list  402  includes a sender column  410 , which identifies the sender of each message. As illustrated in  FIG. 4A , the senders may be identified in various ways. For example, the sender of the third message  424  is identified only by an email address, the sender of the sixth message  430  is the “HR Department,” and the other senders include one or more names of individual people. In some implementations, the number of names displayed depends on the size of the sender column  410 , and additional names may be indicated by an ellipsis, as illustrated by the fourth message  426 . In some implementations, the formatting of names can depend on how the name was formatted in the message received. For example, in the first message  420  and the fifth message  428 , the names appear in first name/last name order, whereas in the second message  422 , the last name appears first. 
     The message list  402  includes a message summary column  412 , which may include the subject line of each message as well as an initial portion of each message, as illustrated by the first message summary  416 . In some implementations, the message summary includes only the subject line of each message. In some implementations, the message summary may include additional information. 
     In some implementations, the message list  402  includes a date/time column  414 , which specifies the date and/or time that each message was received. As illustrated in  FIG. 4A , some implementations display the received time for messages received on the current day and display the received date for messages received earlier than the current day. Typically, the messages are displayed in reverse chronological order, but implementations typically allow a user to sort the messages in other ways (e.g., sort by sender). 
     A display device  208  is inherently limited in size, so the number of messages displayed in the message list  402  is necessarily limited. Scrolling may be necessary for a user to get to a desired message. If the number of messages in the message list is small enough, all of the messages may be able to be displayed. 
       FIG. 4B  illustrates a display of a single electronic message. For example, a user may select a message from the message list  402  in  FIG. 4A  in various ways (e.g., double clicking, tapping, using a menu item or toolbar icon, etc.), and the selected message is shown as in  FIG. 4B . This example interface for a message includes a header area  452 , and a message body  440 . The message header identifies the sender, the recipients, any carbon copy recipients (and optionally blind carbon copy recipients), the subject line, and a list of attachments. Some implementations include other items in the header  452  as well, such as the date/time the message was sent or received. 
     The sample user interface in  FIG. 4B  also includes an annotation region  442 . The example annotation box  442  in  FIG. 4B  enables a user to create a single annotation for a message  224 , but some implementations enable a user to create two or more annotations for the same message  224 . Some implementations provide an annotation type control  444 , enabling the user to specify a type for the annotation. In this example, the annotation type is “task.” Some implementations support a plurality of different annotation types, with each type having a different set of characteristics. For example, some implementations support an annotation type for “notes,” which are not tasks that the user needs to perform. 
     The annotation content box  446  is where the user enters an annotation. In this example, the user has entered “Edit conference presentation by 6:00 tonight.” Some implementations allow a user to specify a date/time for a task in a separate field, which can be tied to a calendar. The annotation box  442  also includes a save button  448 , so the user can explicitly chose to save an annotation. In other implementations, annotations are saved implicitly, so the user does not need to issue an explicit save command. In some implementations, there is a completion control  450  for the user to designate when the task is complete. Some implementations refer to this as a “remove” or “dismiss” control  450  because a task is sometimes removed without being completed (e.g., deleting a task that was input in error or dismissing a task because it is not important). As illustrated below in  FIG. 4C , some implementations also allow a user to specify completion (or dismissal) of a task from the message list  402 . 
     Once the user has created a task annotation for a message, the message list displays the annotation rather than the message summary. This is illustrated in  FIG. 4C . In this example, the user has entered and saved the annotation illustrated in  FIG. 4B  and returned to the message list  402 . Because there is a pending task for the first message  420 , the annotation  462  displays instead of the previous message summary  416 . In some implementations, a task completion control  460  (also referred to as a task dismissal control  460 ) is located next to the task annotation  462 . This enables a user to dismiss a task (e.g., as complete) without opening up the message. In addition, the task completion control  460  visually identifies the first message  420  as having a task that needs to be completed. Some implementations use one or more other visual cues to distinguish between a task and a regular message. For example, some implementations use a different background color or shading for tasks. Some implementations support multiple task priorities, and use different shading to indicate the task priorities. For example, some implementations use no shading for regular messages, yellow shading for regular tasks, and a shade of red for high priority tasks. Once the task is designated as completed (e.g., using the task completion control  460  or the completion control  450 ), the display returns to showing the message summary  416  as illustrated in  FIG. 4A . 
     Some implementations include a task display control  464 , which determines whether tasks are displayed at all. In the example in  FIG. 4C , the control  464  is set to a first state, in which pending tasks have their task annotations displayed (e.g., for the first message  420 ). When the task display control is in a second state (e.g., unchecked), tasks are not displayed. In this second state, the display would return to that shown in  FIG. 4A , even though there is a pending task for the first message  420 . This provides a quick way to toggle between seeing tasks and seeing only the electronic messages. In some implementations, the task display control has more than two states. For example, some implementations support a third state for the task display control in which the display is filtered to just the messages with tasks, and the task annotations are displayed. Some implementations store the state of the task display control  464  as part of the application configuration data  228 . 
       FIGS. 4D and 4E  illustrate two implementations that support messages with two or more annotations. In these two examples, the user has identified three tasks  476 ,  478 , and  480  associated with the fourth message  426 . Each of the tasks has a corresponding task completion control (i.e., the controls  470 ,  472 , and  474 ). In the implementation illustrated in  FIG. 4D , all three tasks are displayed on a single row  426 . If one or two of the tasks are completed, the completed tasks disappear and the remaining tasks are displayed. When all three tasks are completed, the original message summary is displayed. 
     In the implementation illustrated in  FIG. 4E , the three tasks are displayed on separate rows  426 - 1 ,  426 - 2 , and  426 - 3  in the message list. As in  FIG. 4D , if one or two of the tasks are marked as completed, the corresponding rows disappear from the message list. When the last task is completed, the task annotation and corresponding task completion control are replaced by the original message summary. 
       FIG. 4F  illustrates an alternative implementation in which task descriptions are displayed in addition to the message summaries. After a user has created an annotation as illustrated in  FIG. 4B , both the message summary  416  and the annotation  462  are displayed. In some implementations, a task dismissal control  460  is displayed adjacent to the task description  462 , enabling the user to dismiss the task quickly. As noted earlier, a task may be dismissed because it has been completed or for various other reasons (e.g., the user does not have time to complete the task soon, and does not want the task description cluttering the message list  402  in the meantime). 
       FIG. 4G  illustrates an alternative implementation of a user interface that enables a user to provide a task description corresponding to a message. In this implementation, the user interface includes a task description box  472 , which allows the user to enter or edit a task description that describes the message (e.g., what the user needs to do). The content box  474  includes the current text that will be displayed as the message summary when viewing the message list  402 . This allows a user to read a message one time and summarize what the user needs to do. The task description box  472  also includes a save button  476 , which allows the user to save changes, and a dismiss button  478 , which removes the description (thus returning the message summary text to its default state). In  FIG. 4G , the user has set the text to the task description  462 . Once the user saves the task description, the message list displays as shown in  FIG. 4C , with the task description  462  replacing the previous message summary  416 . In some implementations, the message list includes a box or button  460 , which acts as a dismiss button in this context. 
       FIGS. 5A-5C  provide a flowchart of a process  500 , performed by a computing device, for annotating ( 502 ) an electronic message. The method is performed ( 504 ) at a computing device  102  having one or more processors and memory. The memory stores ( 504 ) one or more programs configured for execution by the one or more processors. 
     A user interface for an electronic messaging application is displayed ( 506 ). In some implementations, the electronic messaging application is ( 508 ) an email application. In some implementations, the electronic messaging application is an instant messaging application. The user interface includes ( 510 ) a list of messages received by the user, as illustrated in  FIG. 4A . Each displayed message includes ( 512 ) a respective message summary that is displayed, such as the message summary  416  for the first message  420  in  FIG. 4A . 
     In some implementations, the user interface includes ( 514 ) a task view control  464 , as illustrated in  FIG. 4C . When the task view control  464  is in a first state, task descriptions are displayed ( 516 ) instead of message summaries for each displayed message that has a user-annotated task description, and message summaries are displayed ( 516 ) for each displayed message that has no user-annotated task description. When the task view control is in a second state, message summaries are displayed ( 518 ) for all of the displayed messages regardless of whether any of the displayed messages have user-annotated task descriptions. 
     A description of a first task associated with a first message is received ( 520 ) from the user, as illustrated in  FIG. 4B  above. The user provides ( 522 ) the description using a control in the user interface, such as the annotation content box  446  in  FIG. 4B . In some implementations, the user provides the description by selecting ( 523 ) from a list of task descriptions displayed by the control. For example, the control may generate task description options based on previous task descriptions entered by the user, or a set of predefined task descriptions. In some implementations, the user may select one of the presented task description options or enter a new description from scratch. The description is stored ( 524 ) as a first annotation  226  associated with the first message. The displayed messages in the list of messages are then updated ( 526 ), displaying for the first message the first annotation as a substitute for the message summary of the first message. This was illustrated above in  FIG. 4C   
     In some instances, the user provides ( 528 ) input to dismiss the first task (e.g., using the task completion control  460  or the completion control  450 ). In some instances, the first task is dismissed because it has been completed. In response, the displayed messages in the list of messages are updated ( 530 ), displaying for the first message the message summary of the first message instead of the first annotation. For example, going from  FIG. 4C  back to  FIG. 4A  when the task for the first message  420  is completed. 
     Because task annotations and the status of those annotations are stored (e.g., in the database  116 ), closing the messaging application does not result in the loss of that information. For example, in some instances, a user closes ( 532 ) the messaging application  106 , and subsequently reopens ( 534 ) the messaging application  106 . When the messaging application reopens ( 536 ), the list of messages  402  is displayed ( 536 ), including displaying the first message with the first annotation as a substitute for the message summary of the first message. In some instances, the first message may not be displayed in the message list when the messaging application reopens because the messages are displayed in a specific order and by default the displayed messages are those at the beginning of the list. For example, if the message list can only display 20 messages, and the message with the annotation is the 25th message in reverse chronological order, then the 25th message and its task annotation are not initially displayed. In some implementations, the viewing state in the message list is stored (e.g., as part of the application configuration  228 ), and when the application reopens, the message list automatically scrolls to the same position. 
     In some instances, a user provides one or more additional task descriptions for a single message, as illustrated in  FIGS. 4D and 4E . The one or more additional task descriptions associated with the first message are received ( 538 ) from the user and each description of an additional task is stored ( 540 ) as an additional annotation associated with the first message. In this case, the displayed messages in the list of messages is updated ( 542 ) to display for the first message the first annotation and the additional annotations as a substitute for the message summary of the first message, as illustrated in  FIGS. 4D and 4E . In some implementations, each of the first annotation and additional annotations is displayed ( 544 ) in a separate row in the list of messages, with the respective annotation instead of the message summary of the first message. This is illustrated above in  FIG. 4E . 
     When there is a single annotation for a message, the display of the message returns to normal (i.e., showing the message summary) when the task is designated as completed. When there are multiple tasks associated with a single message, the process is a little different. Consider the case where there are multiple pending tasks for a single message, as illustrated in  FIG. 4D or 4E . At some point the user designates ( 546 ) one or more of the tasks as completed. When one or more of the tasks remain ( 548 ) uncompleted, the user interface displays ( 548 ) for the first message the annotations corresponding to the uncompleted tasks. The annotations for the uncompleted tasks are displayed ( 548 ) instead of the message summary of the first message. When the tasks are all completed ( 550 ), the user interface displays ( 550 ) for the first message the message summary of the first message. 
     The terminology used in the description of the invention herein is for the purpose of describing particular implementations only and is not intended to be limiting of the invention. As used in the description of the invention and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and/or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and/or groups thereof. 
     The foregoing description, for purpose of explanation, has been described with reference to specific implementations. However, the illustrative discussions above are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The implementations described herein were chosen and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to best utilize the invention and various implementations with various modifications as are suited to the particular use contemplated.