Patent Publication Number: US-11042561-B2

Title: Extracting data from communications related to documents using domain-specific grammars for automatic transaction management

Description:
BACKGROUND 
     Related Art 
     The disclosed embodiments relate to techniques for extracting data. More specifically, the disclosed embodiments relate to techniques for extracting data from communications related to documents. 
     Data processing and exchange are essential to many business and personal transactions. For example, small businesses may use accounting and/or inventory data to obtain and share reports regarding inventory sales, customer invoices, and/or cash flow. Similarly, healthcare providers may examine medical records to view patient information related to insurance providers, medical conditions, and/or office visits. 
     In addition, data exchange among users frequently involves the use of documents such as word-processing documents, spreadsheets, and/or Portable Document Format (PDF) documents. For example, a business may manage business transactions with a set of customers by creating a set of bills, invoices, and/or other types of documents containing data associated with the business transactions and transmitting the documents to the respective customers via email. The customers may use the data in the documents to pay the bills and/or invoices, respond to the business, and/or update their records of the transactions. 
     However, information relevant to document-based transactions is frequently found outside of the documents. As a result, a user that relies solely on information from a document to perform a transaction related to the document may not be aware of other options for performing the transaction, which are specified and/or described outside of the document. For example, a business may send an invoice as an attachment in an email to a customer. The business may also include discounts or promotions related to products or services in the invoice and relevant dates (e.g., deadlines, periods of validity, etc.) for the discounts or promotions in the body of the email. If the customer ignores the email&#39;s content and pays the invoice based solely on the amounts and due dates in the invoice, the customer may pay a higher amount than if the customer took advantage of the discounts or promotions in the email. 
     Consequently, use of documents may be facilitated by mechanisms for supplementing data in the documents with relevant data that is found outside of the documents. 
     SUMMARY 
     The disclosed embodiments provide a system that processes data. During operation, the system obtains a communication associated with a document and extracts data associated with the document from the communication. Next, the system uses the extracted data from the communication and document data from the document to build a context associated with the document. The system then uses the context to facilitate use of the document by a user associated with the communication. 
     In some embodiments, the system also uses information from the communication to obtain an additional communication related to the communication, and includes data from the additional communication in the extracted data. 
     In some embodiments, extracting the data from the communication involves applying a domain-specific grammar to the communication to obtain the data from the communication. 
     In some embodiments, using the extracted data and the document data to build the context associated with the communication involves applying the extracted data to the document data to obtain one or more facts and one or more insights, and including the one or more facts and the one or more insights in the context. 
     In some embodiments, using the context to facilitate use of the document by the user involves at least one of:
         (i) providing the context to the user;   (ii) generating a reminder associated with the context;   (iii) recommending a user action based on the context; and   (iv) performing an action for the user based on the context.       

     In some embodiments, the document includes a financial document. 
     In some embodiments, the extracted data includes at least one of a discount, a date, and a promotion. 
     In some embodiments, the communication is at least one of an email, a text message, an instant message, a social-networking post, and a voicemail. 
     In some embodiments, the document is provided using an attachment or a link in the communication. 
    
    
     
       BRIEF DESCRIPTION OF THE FIGURES 
         FIG. 1  shows a schematic of a system in accordance with the disclosed embodiments. 
         FIG. 2  shows the creation of a context associated with a document in accordance with the disclosed embodiments. 
         FIG. 3  shows a flowchart illustrating the processing of data in accordance with the disclosed embodiments. 
         FIG. 4  shows a computer system in accordance with the disclosed embodiments. 
     
    
    
     In the figures, like reference numerals refer to the same figure elements. 
     DETAILED DESCRIPTION 
     The following description is presented to enable any person skilled in the art to make and use the embodiments, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present disclosure. Thus, the present invention is not limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein. 
     The disclosed embodiments provide a method and system for processing data. The data may include document data from a document such as a bill, invoice, form, statement, transcript, record, receipt, and/or tax document. The data may also include data extracted from one or more communications related to the document. For example, the data may be obtained from emails, text messages, instant messages, and/or voicemails and include discounts, promotions, additional terms, verification data, and/or other options related to a transaction associated with the document. As a result, the data may be applied to and/or combined with the document data to improve use of the document data and/or carrying out of a transaction associated with the document by a user. 
     As shown in  FIG. 1 , the data may be included in a document  110  such as a word-processing document, spreadsheet, Portable Document Format (PDF) document, web page, and/or another type of digitized document. For example, document  110  may be a bill, invoice, form, statement, transcript, record, receipt, tax document, and/or other type of document that includes a specific type and/or set of data. Document  110  may be generated digitally using a word-processing application, spreadsheet application, diagramming application, graphics-editing application, and/or other application for creating electronic documents. Alternatively, document  110  may correspond to a scanned and/or photographed digital copy of a physical document. 
     In one or more embodiments, the system of  FIG. 1  includes functionality to automatically extract data from document  110  without requiring use of a template and/or custom code for extracting the data. As shown in  FIG. 1 , the system may correspond to a data-processing framework  102  that provides a document-processing apparatus  106  for obtaining document data  114  from document  110 . For example, document-processing apparatus  106  may analyze the contexts of individual words in document  110  and store data associated with the words and contexts in a set of data elements and tags. As a result, document-processing apparatus  106  may be capable of obtaining document data  114  from document  110  in the absence of a template that specifies the locations of data elements in document  110 . Alternatively, if a template for document  110  is available, document-processing apparatus  106  may use the template to extract document data  114  from document  110 . 
     Document data  114  from document  110  may then be used in a transaction involving two or more users. For example, a business owner may provide an invoice to a customer as an attachment in an email. The customer may download the attachment, open the invoice, and use information in the invoice to pay the business owner for goods or services specified in the invoice. Moreover, if document data  114  has been extracted from the invoice by document-processing apparatus  106 , document data  114  may be provided to a management apparatus  130  in data-processing framework  102  and/or a financial-management application associated with data-processing framework  102 . Management apparatus  130  and/or the application may use document data  114  to generate one or more payment reminders for the invoice and/or schedule a payment of the invoice for the customer. 
     Those skilled in the art will appreciate that additional information related to document  110  may be found in one or more communications  116 - 118  associated with document  110 . The additional information may facilitate use of document  110  and/or carrying out of a transaction associated with document  110 . Continuing with the above example, the business owner may specify additional terms associated with the invoice in the body of the email. The business owner may also inform the customer of discounts or promotions related to the invoice in the same email and/or in a separate email to the customer. Additional discounts, promotions, and/or options associated with payment of the invoice may further be found in emails from other senders, such as “daily deal” websites and/or affiliates. Thus, if the customer pays the invoice based solely on the amounts and/or due dates in the invoice, the customer may pay a higher amount than if the customer took advantage of the discounts or promotions in the email(s). 
     In one or more embodiments, data-processing framework  102  includes functionality to enhance and/or supplement document data  114  with additional data  112  extracted from one or more communications  116 - 118  associated with document  110 . In turn, the additional data  112  may facilitate use of document  110  and/or participation in a transaction associated with document  110  by a user associated with document  110 . 
     More specifically, a communication-processing apparatus  104  may obtain communication  116  associated with document  110  and extract data  112  associated with document  110  from communication  116 . Communication  116  may include an email, text message, instant message, voicemail, and/or other form of digital and/or digitized communication involving two or more users. Communication  116  may be stored in a communication repository  120  with one or more additional communications  118 . For example, communications  116 - 118  may be obtained from an email inbox, a text or instant message thread, a voicemail inbox, cloud storage, and/or a backup mechanism. A crawler associated with communication-processing apparatus  104  may detect the arrival of new communications in communication repository  120 , or the crawler and/or communication-processing apparatus  104  may periodically obtain and process batches of communications from communication repository  120 . 
     Communication-processing apparatus  104  may also use communication  116  to obtain document  110 . For example, communication-processing apparatus  104  may obtain document  110  as an attachment to communication  116 . Alternatively, communication-processing apparatus  104  may access document  110  through a link and/or reference to document  110  in communication  116 . Communication-processing apparatus  104  may provide document  110  to document-processing apparatus  106  for extraction of document data  114  from document  110 , as described above and in the above-referenced application. 
     To extract data  112  from communication  116 , communication-processing apparatus  104  may scan communication  116  to obtain a sender, recipient, and/or document  110  from communication  116 . Communication-processing apparatus  104  may also detect a domain related to communication  116  based on one or more keywords found in communication  116 . For example, communication-processing apparatus  104  may use a natural-language-processing (NLP) technique to parse text in communication  116  and identify the sender, recipient, and/or document  110  associated with communication  116 . Communication-processing apparatus  104  may also match keywords in the parsed sentences to one or more domains (e.g., application domains, business domains, etc.) supported by data-processing framework  102 . If communication  116  is not text-based (e.g., voicemail, video chat, phone call, etc.), communication-processing apparatus  104  and/or another component of data-processing framework  102  may use a speech-recognition technique and/or another mechanism to convert communication  116  into text before scanning communication  116 . 
     Next, communication-processing apparatus  104  may apply a domain-specific grammar (e.g., domain-specific grammar  1   126 , domain-specific grammar z  128 ) to communication  116  to obtain domain-specific words and/or data  112  from communication  116 . For example, communication-processing apparatus  104  may obtain one or more domain-specific grammars for domains matched by keywords in communication  116  from a grammar repository  124 . Communication-processing apparatus  104  may use the domain-specific grammar(s) to detect structures in communication  116  represented by the domain-specific grammar(s) and extract data elements from the structures, as described in further detail below with respect to  FIG. 2 . 
     Communication-processing apparatus  104  may additionally use data  112  and/or other information from communication  116  to identify and/or obtain one or more additional communications  118  related to communication  116 . For example, communication-processing apparatus  104  may obtain additional communications  118  associated with the same sender and/or domain as communication  116 . As with communication  116 , communication-processing apparatus  104  may parse communications  118  and/or apply domain-specific grammars to communications  118  to obtain additional data from communications  118 . Communication-processing apparatus  104  may then supplement data  112  with the additional data. 
     After data  112  is extracted from communication  116  and/or related communications  118 , a context-building apparatus  108  in data-processing framework  102  may build a context  122  associated with document  110  using data  112  and document data  114  from document-processing apparatus  106 . In particular, context-building apparatus  108  may apply data  112  to document data  114  to obtain context  122  as one or more facts and/or one or more insights. For example, context-building apparatus  108  may apply a discount found in communication  116  and/or communications  118  to an amount due in an invoice to obtain a discounted amount for the invoice and/or a pay-by date for the discounted amount. 
     Finally, a management apparatus  130  in data-processing framework  102  may use context  122  to facilitate use of document  110  by a user associated with communication  116 . Management apparatus  130  may provide facts and/or insights from context  122  to the user. For example, management apparatus  130  may inform the user of the facts and/or insights from context-building apparatus  108  in one or more notifications. Management apparatus  130  may also generate a reminder associated with context  122 . For example, management apparatus  130  may remind the user of a due date and/or deadline associated with document  110  prior to the due date and/or deadline. Finally, management apparatus  130  may recommend a user action and/or perform an action for the user based on context  122 . For example, management apparatus  130  may recommend payment of a bill or invoice during a period associated with a discount on the bill or invoice, or management apparatus  130  may automatically schedule a payment to occur within the period to ensure that the user receives the discount. 
     By supplementing document data (e.g., document data  114 ) from documents (e.g., document  110 ) with data (e.g., data  112 ) extracted from communications (e.g., communications  116 - 118 ) related to the documents, data-processing framework  102  may enable better understanding and use of information in the documents by users associated with the documents. In turn, data-processing framework  102  may optimize subsequent communications and/or transactions associated with the documents for the users, thus reducing overhead and/or inefficient handling of the transactions by the users. 
     Those skilled in the art will appreciate that the system of  FIG. 1  may be implemented in a variety of ways. First, communication-processing apparatus  104 , document-processing apparatus  106 , context-building apparatus  108 , management apparatus  130 , and grammar repository  124  may be provided by a single physical machine, multiple computer systems, a grid, one or more databases, one or more filesystems, and/or a cloud computing system. In addition, communication-processing apparatus  104 , document-processing apparatus  106 , context-building apparatus  108 , and/or management apparatus  130  may be provided by the same hardware and/or software component or execute independently from one another. For example, communication-processing apparatus  104 , document-processing apparatus  106 , context-building apparatus  108 , and management apparatus  130  may form a part of an application that is used to process and manage data from a set of documents and related communications for a user. On the other hand, one or more components of data-processing framework  102  may execute outside the application and provide data from the documents and/or communications in a form that can be used by the application and/or other applications after the data is extracted from the documents and/or communications. 
     Second, data-processing framework  102  may be configured for use with various types of electronic documents, communications, and/or data. For example, data-processing framework  102  may accept multiple types and/or file formats of documents and/or include functionality to convert the documents into a file format that is suitable for text and/or data extraction. Along the same lines, data-processing framework  102  may be capable of extracting data from various types of communications, including text-based communications (e.g., emails, text messages, instant messages, social-networking posts, forum messages, etc.) and/or other types of communications (e.g., phone calls, video chats, voicemails, etc.). Data-processing framework  102  may further obtain and/or store data  112 , document data  114 , and/or domain-specific grammars as relational-database records, property lists, Extensible Markup Language (XML) documents, JavaScript Object Notation (JSON) objects, and/or other types of structured data. 
       FIG. 2  shows the creation of a context  212  associated with a document  206  in accordance with the disclosed embodiments. As mentioned above, document  206  may be obtained from a communication  202  such as an email, text message, instant message, social-networking post, and/or voicemail. For example, document  206  may be included in communication  202  as an attachment, or document  206  may be obtained using a link and/or reference in communication  202 . Document data  210  may also be extracted from document  206 , with or without a template for document  206 . 
     Next, communication  202  may be scanned to obtain information such as a sender, recipient, and/or domain  218  associated with communication  202 . For example, an NLP technique may be used to identify the sender, recipient, and/or other entities associated with an email containing an invoice. The NLP technique may also match one or more keywords (e.g., “invoice,” “pay,” “payment,” “business,” etc.) in the email and/or invoice to a domain related to invoice processing and/or financial transactions. 
     A domain-specific grammar  204  for domain  218  may also be applied to communication  202  to obtain data  208  associated with document  206 . For example, a domain-specific grammar for invoice processing may include the following: 
                                INVOICE [       DISCOUNT:x       DATE:d       AMOUNT:a       DISTANCE:r       (Avail x% discount if paid by d)       (Get a discount of a &lt;filler&gt;)       (Obtain a [rebate|discount] of [x%|a] as &lt;filler&gt;) (x% off if paid via         &lt;filler&gt;)       (x% waived &lt;filler&gt; shipping is within r miles)       (No shipping cost&lt;filler&gt;for order [more|greater|higher] than a)       ]                    
As a result, the domain-specific grammar may specify one or more structures that allow data  208  (e.g., discount, date, amount, distance) related to discounts and/or promotions for an invoice to be extracted from text in communication  202 . The domain-specific grammar  204  may then be applied to a sentence such as “Avail 5% discount if paid before Nov. 20, 2013” to obtain a discount of 5% and a date of “Nov. 20, 2013” from the sentence.
 
     Domain  218 , data  208 , and/or other information from communication  202  may further be used to obtain an additional communication  220  related to communication  202 . Domain-specific grammar  204  and/or another domain-specific grammar may be applied to communication  220 , and data extracted from communication  220  may be included in data  208 . If more communications related to communication  202  are found, the domain-specific grammar(s) may also be applied to the communications, and data extracted from the communications may be added to data  208 . 
     Data  208  extracted from communication  202  and/or communication  220  and document data  210  from document  206  may then be used to build context  212 . In particular, data  208  may be applied to document data  210  to obtain one or more facts  214  and one or more insights  216  that are included in context  212 . For example, document data  210  for an invoice may include the following: 
     Vendor name—John&#39;s Furnishings 
     Due Date—Nov. 30, 2013 
     Bill Date—Oct. 20, 2013 
     Amount Due—$546.08 
     Line Item 1—Window blinds $380.00 
     Line Item 2—Cushions $166.08 
     Similarly, data  208  from communication  202  and/or communication  220  may include a discount of 10% and a due date of Nov. 20, 2013. Data  208  may then be applied to document data  210  to obtain an insight containing a discounted amount of $491.48 and a pay-by date for the discounted amount of Nov. 20, 2013. Data  208 , document data  210 , and/or the insight may then be provided to a user in the following form:
         “$546.08 due to John&#39;s Furnishings by Nov. 30, 2013. Pay only $491.48 if paid by Nov. 20, 2013.”
 
Consequently, context  212  may include additional information that is relevant to the invoice but not present in the invoice. The additional information may allow the user to take advantage of a discount that the user may otherwise be unaware of, thus facilitating use and/or payment of the invoice by the user.
       

       FIG. 3  shows a flowchart illustrating the processing of data in accordance with the disclosed embodiments. In one or more embodiments, one or more of the steps may be omitted, repeated, and/or performed in a different order. Accordingly, the specific arrangement of steps shown in  FIG. 3  should not be construed as limiting the scope of the embodiments. 
     Initially, a communication associated with a document is obtained (operation  302 ). The communication may be an email, a text message, an instant message, a social-networking post, a voicemail, and/or another type of digital or digitized communication. Next, data associated with the document is extracted from the communication (operation  304 ). For example, a domain-specific grammar may be applied to the communication to identify structures in the communication and obtain data from the structures. 
     An additional communication related to the communication may be available (operation  306 ). For example, the additional communication may be obtained using information (e.g., sender, recipient, domain, keywords, etc.) and/or data from the communication. If an additional communication is available, data from the additional communication is included in the extracted data (operation  308 ). For example, the domain-specific grammar may be applied to the additional communication to obtain additional data related to the communication and/or document. If no additional communication is available, no additional data is included in the data extracted from the communication. 
     The extracted data and document data from the document are then used to build a context associated with the document (operation  310 ). For example, the extracted data may be applied to the document data to obtain one or more facts and one or more insights, and the fact(s) and/or insight(s) may be included in the context. 
     Finally, the context is used to facilitate use of the document by a user associated with the communication (operation  312 ), such as a recipient of the communication. To facilitate use of the document by the user, the context may be provided to the user to notify the user of the facts and/or insights obtained from the extracted data and/or document data. For example, the user may be notified of available discounts or promotions, errors or discrepancies, and/or changes in terms or additional terms associated with the document. A reminder associated with the context may also be generated to allow the user to take advantage of the insights in a timely manner. For example, a reminder may be generated to remind the user of a pay-by date for a discount or promotion. Finally, the context may be used to recommend a user action and/or perform an action for the user. For example, the user may be provided with a phone number and/or email address for contacting another user associated with the document or communication if an error or discrepancy is found in the document or communication. Alternatively, an email requesting review or explanation of the error or discrepancy may be automatically generated and sent to the other user on the user&#39;s behalf. 
       FIG. 4  shows a computer system  400  in accordance with the disclosed embodiments. Computer system  400  includes a processor  402 , memory  404 , storage  406 , and/or other components found in electronic computing devices. Processor  402  may support parallel processing and/or multi-threaded operation with other processors in computer system  400 . Computer system  400  may also include input/output (I/O) devices such as a keyboard  408 , a mouse  410 , and a display  412 . 
     Computer system  400  may include functionality to execute various components of the present embodiments. In particular, computer system  400  may include an operating system (not shown) that coordinates the use of hardware and software resources on computer system  400 , as well as one or more applications that perform specialized tasks for the user. To perform tasks for the user, applications may obtain the use of hardware resources on computer system  400  from the operating system, as well as interact with the user through a hardware and/or software framework provided by the operating system. 
     In one or more embodiments, computer system  400  provides a system for processing data. The system may include a communication-processing apparatus that obtains a communication associated with a document and extracts data associated with the document from the communication. The system may also include a context-building apparatus that uses the extracted data from the communication and document data from the document to build a context associated with the document. Finally, the system may include a management apparatus that uses the context to facilitate use of the document by a user associated with the communication. 
     In addition, one or more components of computer system  400  may be remotely located and connected to the other components over a network. Portions of the present embodiments (e.g., communication-processing apparatus, context-building apparatus, management apparatus, etc.) may also be located on different nodes of a distributed system that implements the embodiments. For example, the present embodiments may be implemented using a cloud computing system that extracts data from communications related to a set of documents and uses the extracted data to facilitate use of the documents by a set of remote users. 
     The data structures and code described in this detailed description are typically stored on a computer-readable storage medium, which may be any device or medium that can store code and/or data for use by a computer system. The computer-readable storage medium includes, but is not limited to, volatile memory, non-volatile memory, magnetic and optical storage devices such as disk drives, magnetic tape, CDs (compact discs), DVDs (digital versatile discs or digital video discs), or other media capable of storing code and/or data now known or later developed. 
     The methods and processes described in the detailed description section can be embodied as code and/or data, which can be stored in a computer-readable storage medium as described above. When a computer system reads and executes the code and/or data stored on the computer-readable storage medium, the computer system performs the methods and processes embodied as data structures and code and stored within the computer-readable storage medium. 
     Furthermore, methods and processes described herein can be included in hardware modules or apparatus. These modules or apparatus may include, but are not limited to, an application-specific integrated circuit (ASIC) chip, a field-programmable gate array (FPGA), a dedicated or shared processor that executes a particular software module or a piece of code at a particular time, and/or other programmable-logic devices now known or later developed. When the hardware modules or apparatus are activated, they perform the methods and processes included within them. 
     The foregoing descriptions of various embodiments have been presented only for purposes of illustration and description. They are not intended to be exhaustive or to limit the present invention to the forms disclosed. Accordingly, many modifications and variations will be apparent to practitioners skilled in the art. Additionally, the above disclosure is not intended to limit the present invention.