Abstract:
Exemplary embodiments of the present disclosure are directed towards a system for processing communications that detects just the portions of the communication requesting action, a response, or increased attention from a user, wherein said system comprises: (a) a message filter unit that analyzes the content and metadata of messages conveyed by various communication modalities and determines which portions of the messages request action, a response, or increased attention from the user; (b) a sender importance unit that determines from past communication patterns the perceived urgency that the user will afford to a new message from a particular sender; and (C) a user interface unit that alerts the user to detected items that require attention, response or action. Additionally, the disclosure describes a method for managing a list of tasks requiring attention automatically, where incoming messages are scanned and action items extracted and added to the list.

Description:
TECHNICAL FIELD 
       [0001]    The subject matter generally relates to a system and method for automatically mining corpora of communications and identifying messages or phrases that require the recipient&#39;s attention, response or action. 
       BACKGROUND 
       [0002]    In general, a user device operating in a data communication network is configured with various communication modalities (e.g., SMS applications, Email applications, Social Networking applications, Calendar applications, and other applications). The user device is bombarded with multiple messages across these communication modalities. Some of these messages may require a user&#39;s prompt attention, some may not need prompt attention, and some may not require any attention. Determining the importance of received messages and identifying the messages that require user attention is difficult. It is desirable to determine the importance of the received messages and notify the user of important messages. 
         [0003]    Furthermore many messages, such as marketing and promotional messages, associated with the aforementioned communication modalities try to assume familiarity and demand responses from the user in a way confusingly close to legitimate requests for expertise and attention. 
         [0004]    Therefore, it is desirable to have a system and method that ascertains the necessity of requesting user attention, and tracks and prioritizes the messages requiring user attention and user response. 
       BRIEF SUMMARY 
       [0005]    The following presents a simplified summary of the disclosure in order to provide a basic understanding to the reader. This summary is not an extensive overview of the disclosure and it does not identify key/critical elements of the invention or delineate the scope of the invention. Its sole purpose is to present some concepts disclosed herein in a simplified form as a prelude to the more detailed description that is presented later. 
         [0006]    A more complete appreciation of the present invention and the scope thereof can be obtained from the accompanying drawings that are briefly summarized below and the following detailed description of the presently preferred embodiments. 
         [0007]    Exemplary embodiments of the present disclosure are directed towards a system and method for automatically mining corpora of communications and identifying messages or phrases that require the recipient&#39;s attention, response or action. 
         [0008]    According to one or more exemplary embodiments, the method for automatically mining a corpus of communications and identifying critical messages may be performed locally with a data-communication-device-based approach, performed centrally with a server-unit-based approach or may be configured to operate between one or more data communication devices, with a client-server architecture wherein the client device may be any data communication device operated in a data communication network (e.g., a server, client device, or even a router). 
         [0009]    A preferred aspect of the present disclosure is to automatically review a user&#39;s incoming corpus of communications and extract those communications that require a response, extra attention, or follow up of action from the user. 
         [0010]    A preferred aspect of the present disclosure is to mine data from multiple communication modalities such as email, SMS, instant messaging, social networking applications sites such as Facebook and Twitter, phone voice mail communications, audio and video streams and other similar modalities configured in the data communication device. 
         [0011]    Another preferred aspect of the present disclosure is to split the extracted data from each communication modality into multiple phrases. 
         [0012]    A preferred aspect of the present disclosure is to base the system on classification algorithms that extract features from message content, message metadata, the user&#39;s contact list and communication history. In one embodiment, the classification algorithm is a supervised machine-learning algorithm that may use, but is not limited to, the Bayesian combination of probabilities. 
         [0013]    Also another preferred aspect of the present disclosure is to highlight the corresponding processed phrases that do contain an actionable item, a question requiring response, or message needing extra attention over the user interface of the data communication device of the user. 
         [0014]    Another preferred aspect of the present disclosure is to use exemplar-based, nearest-neighbor based on the cosine distance between vectors representing phrases and prototypical examples. 
         [0015]    Also, another preferred aspect of the present disclosure is to include message content features such as (without limitation) n-grams of consecutive words, and the presence and position of key words (e.g., “please” or “ASAP”). 
         [0016]    A preferred aspect of the present disclosure is to include message metadata features such as (without limitation) message length, time and date of sending, headers included from delivery services (e.g., spam-filter ratings), number and identities of other recipients, whether the recipient is specifically named or included as part of a mailing list or whether the message was in response to a previous message. 
         [0017]    Another preferred aspect of the present disclosure is to include any or all of the user&#39;s contact lists, such as an email address book, social network contacts, phone numbers in mobile phone, users sharing a corporate email domain, contacts who have previously received mail from the user, or the transitive closure (whether limited to a certain number of degrees or unlimited) of such trusted contacts. 
         [0018]    Also another preferred aspect of the present disclosure is to include any or all of the user&#39;s communication history, such as past emails sent and received, past text messages sent and received, past phone calls placed or received, past social media posts or messages sent or received. 
         [0019]    A preferred aspect of the present disclosure is to include steps to transform the phrase into a “canonical” form, which renders consistent forms such as consistent form of contractions and abbreviations, syntactic transformation to handle active/passive voice, syntactic transformation to handle prepositional movement at sentence end (for example “By when is the report due?”=&gt;“When is the report due by?”), conflation of synonyms into an abstract conceptual representation, removal of words unlikely to bear on a message&#39;s need for action/response, including (without limitation): articles, adjectives, excerpts of previous messages forwarded by the sender, directly quoted passages, headers or other materials, social niceties and abstraction of the specific identity of proper nouns, dates or times, places, or numbers. 
         [0020]    Another preferred aspect of the present disclosure is to present the extracted action items in convenient form and a convenient time by a user. 
         [0021]    Another preferred aspect of the present disclosure is to include presentation such as visual highlighting of extracted action item(s), audio summary of extracted action item(s) and entry of extracted action item onto the user&#39;s “Tasks Requiring Attention”. 
         [0022]    Another preferred aspect of the present disclosure is to present notifications to the user based on the context of the user, including, without limitation, information derived from the user&#39;s calendars and sensors such as those in a vehicle, residence, communication device or wearable device. Such sensors could beneficially provide the user&#39;s current location and the speed at which the user is travelling, among other quantities. 
         [0023]    Also another preferred aspect of the present disclosure is to present the user with assistance to reply/handle the extracted action item. 
         [0024]    Further, another preferred aspect of the present disclosure is to provide the user with canned responses that offer a quick response that syntactically matches the form of the question or mention when a real response can be expected. 
         [0025]    Still another preferred aspect of the present disclosure is to analyze templates or past responses from the user that are relevant to the request, and then present them for sending or editing. 
         [0026]    Also a preferred aspect of the present disclosure is to track the completion status of requests extracted from incoming messages. The system also adds extracted items to a representation of tasks requiring attention; such representation may be a “Tasks Requiring Attention” list. The system also controls the presentation of this list and the removal of items from it. 
         [0027]    Another preferred aspect of the present disclosure is to remove items from the representation of tasks requiring attention when the user replies to the corresponding message. 
         [0028]    Also, another preferred aspect of the present disclosure is to enable the removal of items from the representation of tasks requiring attention only if the content of message appears to be a resolution (and not, for example, a request for more time). 
         [0029]    Still another preferred aspect of the present disclosure is to remove items from the representation of tasks requiring attention when the system does not receive responses regarding those items for a certain amount of time. 
         [0030]    Yet another preferred aspect of the present disclosure is to prioritize the order of presentation of action items by any or all of: importance of sender, stated urgency of request and time since request was received. 
         [0031]    Another preferred aspect of the present disclosure is to manage the full cycle of communications that include action items: determining actionability by extracting relevant input features from metadata and content, transforming extracted content, assessing desired output features, alerting the user, supporting the user in completing the action item and supporting the user in tracking completion status/pending items. 
         [0032]    System and method for processing communications that detects just the portions of the communication requesting action, a response, or increased attention from a user are disclosed. The system comprising a message filter unit that analyzes the content and metadata of messages conveyed by various communication modalities and determines which portions of the messages request action, a response, or increased attention from the user. 
         [0033]    The system further includes a sender importance unit that determines from past communication patterns the perceived urgency that the user will afford to a new message from a particular sender. 
         [0034]    The system further includes a user interface unit that alerts the user to detected items that require attention, response or action. 
     
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         [0035]    Other objects and advantages of the present invention will become apparent to those skilled in the art upon reading the following detailed description of the preferred embodiments, in conjunction with the accompanying drawings, wherein like reference numerals have been used to designate like elements, and wherein: 
           [0036]      FIG. 1  is a block diagram depicting a system for automatically mining corpora of communications and identifying messages or phrases that require the recipient&#39;s attention, response, or action, in accordance with exemplary embodiments of the present disclosure. 
           [0037]      FIG. 2  is a diagram depicting a filter module with sub filters for mining corpora of communications and identifying messages or phrases which require the recipient&#39;s attention, response, or action, in accordance with exemplary embodiments of the present disclosure. 
           [0038]      FIG. 3  is a diagram depicting a system for displaying current notifications on the data communication device, in accordance with exemplary embodiments of the present disclosure. 
           [0039]      FIG. 4  is a block diagram depicting a system for assisting a user in responding to or handling action items and tracking completion status, in accordance with exemplary embodiments of the present disclosure. 
           [0040]      FIG. 5  is a flow diagram depicting a method for automatically mining corpora of communications and identifying actions, in accordance with exemplary embodiments of the present disclosure. 
       
    
    
     DETAILED DESCRIPTION 
       [0041]    It is to be understood that the present disclosure is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the drawings. The present disclosure is capable of other embodiments and of being practiced or of being carried out in various ways. Also, it is to be understood that the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. 
         [0042]    The use of “including”, “comprising” or “having” and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. The terms “a” and “an” herein do not denote a limitation of quantity, but rather denote the presence of at least one of the referenced item. Further, the use of terms “first”, “second”, and “third”, and the like, herein do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. 
         [0043]    Referring to  FIG. 1  is a diagram  100  depicting a system for automatically mining corpora of communications and identifying actions, in accordance with exemplary embodiments of the present disclosure. The diagram  100  includes various communication modalities  101 , that may include, but are not limited to, social networking applications, a communication access unit (with the ability to read current and historical messages, email, call logs, voice mails, and mms), a contact list, a location history unit and the like. 
         [0044]    The various communication modalities  101  may be used to identify the user specific contacts, creation date of contacts, recency of last contact, shared domain (which, if it is not a common email provider such as gmail, yahoo, hotmail, etc., may indicate a shared employer or academic institution), and shared last name. Features not available directly from the contact book but require extraction from the call logs may also be included, such as information relating to frequency and length of communication, along with time of first contact and most recent contact, and the like. These data items, collectively called the “metadata” associated with the messages, are inputs that help to evaluate the importance of the message or its sender. 
         [0045]    As shown in  FIG. 1 , the system  100  includes a communication importance estimate unit  102  that may be configured to evaluate content associated with the corpora of communications retrieved from various communication modalities  101 . Estimates of importance may be based on whether the message was responded to, how quickly, by how many recipients, and the amount of discussion that followed. 
         [0046]    As shown in  FIG. 1 , the system  100  includes a sender importance unit  103  that may be configured to process the output received from the communication importance estimate unit  102  and infer the likely importance of each sender of existing messages, and transmit the results to a filter module  105 . The sender importance unit  103  may be modified by user prioritization preferences  104 , such preferences may reflect the times of day when a user is willing to handle work-related messages or people whose messages merit extra consideration, such as a family member. The filter module  105  may be used to filter the corpora of incoming communications, identifying those phrases or messages that require the recipient&#39;s attention, response or action. 
         [0047]    Referring to  FIG. 2  is a diagram  200  depicting a filter module  105  (shown in  FIG. 1 ) with sub filters for mining corpora of communications and identifying messages or phrases that require attention, a response, or action, in accordance with exemplary embodiments of the present disclosure. The filter module  105  may include a message filter  201 , configured to filter the corpora of communications. Filtering the corpora of communications may include a step of excluding communications received from unknown senders and considering only the communications from known senders. For example, known senders may include, but are not limited to, the senders for whom previously a communication has been made through email or SMS, whose identity is listed in the “cc” field in any previous email sent or previously listed as a recipient of SMS, or whose identity is listed as a co-recipient with the user in an email or SMS. Further, the message filter unit  201  may exclude communications by identifying the sender as a promoter or marketer. Identifying the promoters may include a step of identifying if the communication has a different “reply-to” than “from” field, identifying keywords such as “do-not-reply” or “unsubscribe” in the sender&#39;s email address, identifying a known list server (e.g. MailChimp, Convio, ConstantContact, VerticalResponse, Flonetwork, or ExactTarget) in the return path of the sender&#39;s communication. The message filter  201  may also exclude communications containing a “List Unsubscribe” mail header or similar phrase (e.g., “If you cannot view” or “Click here to unsubscribe”) 
         [0048]    As shown in  FIG. 2 , the filter module  105  may include a relevant content filter unit  202  that receives the corpora of communications from the message filter unit  201 . The relevant content filter unit  202  may be configured to remove signatures associated with the communication, bypass excerpts of replies and forwarded communications contained within the communication and extract only the relevant content from the filtered content. The relevant content filter unit  202  may exclude signatures and/or footers associated with the content received from the message filter unit  201  by identifying keywords or phrases such as “If you have received this in error . . . ” or other data elements common to automatically appended signatures including the email address, phone number, job title, fax number, Twitter handle, etc. The relevant content filter unit  202  may also exclude messages sent by auto-responders, as determined by measuring the response time between message arrival and reply arrival and looking for keywords that are commonly found in “out-of-office” messages. The relevant content filter unit  202  also excludes headers that assist with mail delivery protocols and forwarded content, demarcated by phrases such as “Begin forwarded message” or other patterns commonly used to indicate included content, such as “&gt;&gt;” at the beginning of the line. 
         [0049]    As shown in  FIG. 2 , the filter module  105  may include a message segmenter unit  203  configured to collect phrases of filtered content as received from the relevant content filter unit  202 . The message segmenter unit  203  may be configured for converting and dividing the filtered content into multiple phrases such as sentences or other meaningful content units, without limiting the scope of the disclosure. 
         [0050]    As shown in  FIG. 2 , the filter module  105  may include a phrase filter unit  204  configured for receiving the multiple phrases as defined by the message segmenter unit  203 . The phrase filter unit  204  may be configured to filter the phrases defined by the message segmenter unit  203  to make a first pass at eliminating the content that does not require a user&#39;s response, attention, or action, while passing through phrases where the resolution is not easily determined and requires further analysis. The phrase filter unit  204  may be configured to include phrases that have potentially actionable words such as “please” or “send me” or “What time” or phrases that start with a verb (after removing an initial proper name and “please”, if either or both exist); exclude phrases that look like social niceties (e.g., “How are you?” or “How was your weekend?”); determine whether the phrase is too short or too long based on the word count and whether the phrase has too many capitalized words or is in ALL CAPS; exclude phrases that look like rhetorical questions (e.g., “How great is that?”). 
         [0051]    As shown in  FIG. 2 , the filter module  105  may include a canonicalizer unit  205  configured for receiving the filtered phrases from the phrase filter unit  204  and converting variations of the same expressions of the filtered phrases into a single form. The canonicalizer unit  205  may be configured for removing stop words such as articles; performing contraction expansion, including those with omitted apostrophes (such as “haven&#39;t”); abstracting urls, phone numbers, dates, addresses, and names associated with the filtered phrases, so that the canonical form reads just “Call me at PHONE-NUMBER” instead of “Call me at 212-555-1234”; aliasing i.e. converting several different ways of expressing the same sentiment into a single common form, so that splintered data can be aggregated (“I would like to”, “I want to”), many ways to say “please” such as “If you get a chance, would you.” or “would you be so kind as to . . . ”; and removing direct quotations embedded within the filtered phrases. By applying these processes the canonicalizer unit  205  generates canonicalized phrases. 
         [0052]    As shown in  FIG. 2 , the filter module  105  may include a feature extractor unit  206  for receiving the canonicalized phrases generated by the canonicalizer unit  205  and for converting canonicalized phrases into a feature vector. The feature extractor unit  206  determines the length of canonicalized phrases and, for example, sees if (a) “Please” is first word of phrase; (b) “Please” is in the phrase, but not the first word; (c) if the phrase starts with an interrogative word (e.g. Which, where, what, how, why); (d) phrase starts with a 2nd person verb (e.g., “Put”, “Send”, “Pick”, “Go”) or other specific keywords or tokens such as URL&#39;s or phone numbers. The words in the canonicalized phrase may also be converted into n-grams that are extracted as features if they appear in a dictionary of sufficiently common word combinations in the native language. 
         [0053]    As shown in  FIG. 2 , a classifier unit  207  receives the feature vectors generated by the feature extractor unit  206 . The classifier unit  207  may be configured using one or more of a variety of classification techniques to determine actionable content from the received feature vectors. One preferred approach to configuring the classifier unit  207  is to apply supervised machine learning techniques to train the classifier on known positive instances (phrases requiring a recipient&#39;s attention, response, or action) and negative instances (sample phrases not requiring a recipient&#39;s attention, response, or action). The classifier unit  207  may include, but is not limited to, a Naive Bayes Classifier. Each feature in the feature vector is considered in turn with respect to each label (“actionable”, “not actionable”). The predictive power for the presence of that feature is the logarithm of the ratio of instances having both that feature and the label to those instances that have just the label. The scores of all of the features are summed and if the sum for the features deemed “actionable” minus the sum of the same features in the “not actionable” context exceeds a threshold value set during the training phase, the phrase is classified as one requiring user attention, response, or action. 
         [0054]    Referring to  FIG. 3  is a diagram  300  depicting a system for displaying current notifications on the data communication device, in accordance with exemplary embodiments of the present disclosure. The notifications may be presented to the user based on a current user context  310  and user preferences  312 , and the output of the system for automatically mining corpora of communications and identifying messages or phrases which require the recipient&#39;s attention, response, or action  100 . 
         [0055]    As shown in  FIG. 3 , a system for automatically mining corpora of communications and identifying messages or phrases which require the recipient&#39;s attention, response, or action  100  (as shown in  FIG. 1 ) determines which parts of the incoming messages are candidates for being displayed as a current notification on the user&#39;s device. 
         [0056]    As shown in  FIG. 3 , an activity detection unit  311  may be configured for collecting user context information  310  that may include, but is not limited to, sensor data from the user&#39;s communication or other wearable (smart watch, eye piece display, or other personal computing device with limited screen display) or implanted computing devices, or sensors in the user&#39;s vehicle, residence, or office that may be available to the system. These sensors may provide location, speed of travel, lighting conditions, ambient sound, etc. and calendar information (current location information, number and identities of other people present at the location, and scheduled activity). The user preferences  312  may be used for determining how or whether a user would like to receive a notification based on an inferred user activity. For example, a user who is in a meeting might wish to be informed via a vibration and short text message, whereas a user who is driving might prefer an audio summary. A user who is at an office may prefer to see the full text of the message with visual highlighting (e.g., black text on a yellow background) call attention to the phrases in the message requiring the recipient&#39;s attention, response, or action  100 . A user who is away from the office due to travel may want the discovered items to be forwarded via email to his or her assistant or other delegate to be handled in the user&#39;s absence. 
         [0057]    As shown in  FIG. 3 , the importance of each sender is recovered from the sender importance unit  302  The combination of the output of the system for automatically mining corpora of communications and identifying messages or phrases which require the recipient&#39;s attention, response, or action  100 . and the importance of the sender  302 , determines whether this particular message merits the user&#39;s attention. If it does, a request for user attention  301  is generated. The prioritizing unit  303  processes the request for user attention  301 , and information pertaining to the user&#39;s availability that is used to generate current notifications  305  and suppressed notifications  304 . The prioritizing unit  303  may also be configured for receiving queued notifications and storing them in a queued notifications repository unit  306 . 
         [0058]    As shown in  FIG. 3 , an alert generating unit  307  receives the current notifications generated by the prioritizing unit  303  and displays the current notifications on the user interface of the data communication device  308  of the user. The user&#39;s response to that notification is one or more user events  309  which may update the user preferences  312 . 
         [0059]    Referring to  FIG. 4  is a diagram  400  depicting a system for assisting a user in responding to or handling action items and tracking completion status. 
         [0060]    As shown in  FIG. 4 , a reply generating unit  403  may be configured to generate possible replies to the action item based on the content of the action item  401 , past replies of the user, user preferences  402  and the like. 
         [0061]    As shown in  FIG. 4 , the system may include a representation of tasks that may require the user&#39;s attention  404 , e.g., a “Tasks Requiring Attention”. The representation of tasks that require attention includes each of the items that requires a user&#39;s action, along with the person requesting the action and the date by that it must be accomplished (the deadline) if mentioned. The task removal unit  405  may be configured to manage removal of tasks from that list automatically, based on specific user actions or system inferences. Example user actions include:
       a) The user makes a non-trivial response to the message   b) The user explicitly checks off the item   c) The user communicates with the originator of the item by a different medium (e.g., send an SMS in reply to an email)   d) The user travels to a location where the task could be completed
 
The system might infer that an item can be removed if:
   a) The message contains a deadline (e.g., “Please RSVP before Tuesday if you plan to attend.”) which has already passed.   b) The user has established a default deadline (e.g., 48 hours from receipt of the message) that has already passed.       
 
         [0068]    Referring to  FIG. 5  is a flow diagram  500  depicting a method for automatically mining a corpus of communications and identifying actions, in accordance with exemplary embodiments of the present disclosure. The method starts at step  501 , a communication importance-estimating unit configured to retrieve a corpus of communications from various communication modalities. The content of the various communication modalities may be evaluated by the communication importance-estimating unit at step  502 . At step  503 , a sender importance unit is configured to process the output received from the communication importance-estimating unit. The received output is transmitted to the filter module (as described in  FIG. 2 ) for filtering the various communication modalities at step  504 . Further at step  505 , alerts may be displayed on the user interface of the data communication device based on the filtering by an alert generating unit. At step  506 , assistance is provided to the user to reply or handle action items and to track pending or completion status of action items, including addition to the user&#39;s representation of tasks that require attention, if appropriate. 
         [0069]    The claimed subject matter has been provided here with reference to one or more features or embodiments. Those skilled in the art will recognize and appreciate that, despite of the detailed nature of the exemplary embodiments provided here; changes and modifications may be applied to said embodiments without limiting or departing from the generally intended scope. These and various other adaptations and combinations of the embodiments provided here are within the scope of the disclosed subject matter as defined by the claims and their full set of equivalents.