Abstract:
Our approach seeks to reduce the complexity of this type of calculation through approximation and pre-computation. It is designed to work efficiently with modern relational database constructs for content management. The approach is designed to enable the kinds of highly interactive data-driven visualizations that are the hallmark of third generation business intelligence.

Description:
TECHNICAL FIELD 
   The present disclosure generally relates to the field of document analysis, and more particularly to a method for generating an ordered list of signature terms occurring in at least a first document and a second document. 
   BACKGROUND 
   Traditional Term Frequency Inverse Document Frequency approaches are computationally intensive methods for identifying significant terms and phrases that differentiate specific documents from the rest of the corpus. 
   In finding unique terms and phrases for a set of search results, all of the documents in the result set are summed to create a result set frequency, this set is then sorted, and the top N terms or phrases are selected as the signature terms. 
   SUMMARY 
   Our approach seeks to reduce the complexity of this type of calculation through approximation and pre-computation. It is designed to work efficiently with modern relational database constructs for content management. The approach is designed to enable the kinds of highly interactive data-driven visualizations that are the hallmark of third generation business intelligence. 
   It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not necessarily restrictive of the present disclosure. The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate subject matter of the disclosure. Together, the descriptions and the drawings serve to explain the principles of the disclosure. 

   
     BRIEF DESCRIPTION OF THE DRAWINGS 
     The numerous advantages of the disclosure may be better understood by those skilled in the art by reference to the accompanying figures in which: 
       FIG. 1A  is a flow diagram illustrating a method for generating an ordered list of signature terms occurring in at least a first document and a second document; 
       FIG. 1B  is a flow diagram continuing the method illustrated in  FIG. 1A ; and 
       FIG. 1C  is a flow diagram continuing the method illustrated in  FIGS. 1A and 1B . 
   

   DETAILED DESCRIPTION 
   Reference will now be made in detail to the subject matter disclosed, which is illustrated in the accompanying drawings. 
   We begin in the same manner as traditional TFIDF by computing the DF. We then sort the lists to bring the signature terms of the document to the front. 
   We then approximate this list by setting the top Q entries of the TFIDF to 1 and the rest to 0. We can make this approximation because of the Zipf&#39;s Law distribution of terms and phrases in English documents. This is additionally helpful from a computational standpoint because it moves the computation space out of floating point math and replaces it with simple sums over integers. 
   We then truncate the zero entries, leaving just the top terms. At this point we could also use a thesaurus for further term reduction. 
   This technique is most useful in cases where a novel set of results is generated as part of a user interaction. This is very typically the result when a search query is issued; a set of results is returned that a user may want to explore and understand. Given these search results, which constitute a document set within the corpus, we can take the union of all the sets of “indicative terms” for those documents. We then return the top M of them when requested. These are sorted to produce an ordered list of the signature terms of a result set. 
   This computation by contrast is order q*(k+1) for a result set of size k and a threshold of q for the number of entries considered. This dramatic reduction in computation enables functionalities including 3rd generation business intelligence which require real-time interactivity. 
   Referring to  FIG. 1 , a method for generating an ordered list of signature terms occurring in at least a first document and a second document is described. First, a frequency of a first term occurring in a first document is computed,  110 . Then, a frequency of a second term occurring in the first document is computed,  120 . The first term and the second term are sorted to identify the most frequently occurring term in the first document,  130 . A frequency of a third term occurring in a second document is computed,  140 . A frequency of a fourth term occurring in the second document is computed,  150 . The third term and the fourth term are sorted to identify the most frequently occurring term in the second document,  160 . The sorted list for the first document is truncated by selecting a number of terms to retain from the sorted list for the first document,  170 . The sorted list for the second document is truncated by selecting a number of terms to retain from the sorted list for the second document,  180 . The size of at least one of the sorted list for the first document and the sorted list for the second document is reduced by utilizing a thesaurus to identify like terms,  190 . Then, the sorted list for the first document and the sorted list for the second document are combined utilizing a union operation,  200 . Finally, the combined list generated by the union operation is sorted to create an ordered list of signature terms,  210 . 
   In the present disclosure, the methods disclosed may be implemented as sets of instructions or software readable by a device. Further, it is understood that the specific order or hierarchy of steps in the methods disclosed are examples of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the method can be rearranged while remaining within the disclosed subject matter. The accompanying method claims present elements of the various steps in a sample order, and are not necessarily meant to be limited to the specific order or hierarchy presented. 
   It is believed that the present disclosure and many of its attendant advantages will be understood by the foregoing description, and it will be apparent that various changes may be made in the form, construction and arrangement of the components without departing from the disclosed subject matter or without sacrificing all of its material advantages. The form described is merely explanatory, and it is the intention of the following claims to encompass and include such changes.