Abstract:
The present invention makes legal research more efficient by selecting clusters in response to the behavior of a user (e.g., a legal professional such as a paralegal, lawyer, or judge). The clusters, which are formed prior to the user accessing a legal document (and thus, providing user behavior to a system), are identified to the based upon a set of metadata associated with the legal document. At least two clusters are identified and a signal associated therewith is transmitted to the user. Each cluster is associated with a unique legal topic. Further, each cluster may comprise primary and/or secondary authority.

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
       [0001]    This non-provisional patent application claims priority to U.S. Provisional Patent Application Ser. No. 61/217,522 filed Jun. 1, 2009, the contents of which are incorporated herein in entirety. This application is also claims priority to, and is a continuation in part application of, U.S. patent application Ser. No. 11/538,749 filed Oct. 4, 2006 entitled “Systems, Methods, and Software for Identifying Relevant Legal Documents” (now Publication No. U.S. 2008/0033929 A1), which claims priority to U.S. Provisional Patent Application No. 60/723,322 filed Oct. 4, 2005, the collective contents of all of which are incorporated herein in their entirety. 
     
    
     COPYRIGHT NOTICE AND PERMISSION 
       [0002]    A portion of this patent document contains material subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyrights whatsoever. The following notice applies to this document: Copyright © 2010, Thomson Reuters Global Resources. 
       TECHNICAL FIELD 
       [0003]    The present invention relates to systems, methods and interfaces for providing information in response to a computerized search for legal content. 
       BACKGROUND 
       [0004]    The American legal system, as well as some other legal systems around the world, relies heavily on a concept know as stare decisis. This Latin phrase means as “to stand by things decided.” The phrase “stare decisis” is itself an abbreviation of the Latin phrase “stare decisis et non quieta movere” which means “to stand by decisions and do not move that which is quiet.”The American legal system practices stare decisis by deciding similar cases in similar fashion and not overruling previously established law absent a good reason to do so. 
         [0005]    Legal professionals, such as paralegals, judges and lawyers, constantly apply the principle of stare decisis by using legal content to support their arguments and positions. In general, there are two types of sources of legal content, primary sources and secondary sources. Primary sources are judicial opinions dealing with the same legal issue. Primary sources are said to be “binding” if the primary source is from a higher court (or the same exact court) than the court currently deciding the legal issue and the higher court is in the “chain of command” of the court currently deciding the issue. For example, the U.S. Supreme Court&#39;s opinions are binding on all other courts deciding the same issue. However, a federal district court&#39;s opinion from, e.g., New York, is not binding upon a federal district court in Pennsylvania deciding the same issue. The opinion of the New York court is considered persuasive (but not binding) primary authority. Secondary sources comprise other legal content such as law review and other scholarly articles briefs, motions, and administrative decisions. 
         [0006]    Ideally, legal professionals would always be able to support their positions and arguments with primary authority that is binding. However, this is not always practical due to many reasons including, for example, new issues arising in the law which courts have yet to address. Further, parties may also have different interpretations of how or if a particular previous judicial opinion applies to their dispute. In fact, parties may not even agree upon what the legal issue that needs to be resolved. 
         [0007]    In order to provide clients with fast and superior legal services, many legal professionals use computerized research to attempt to find primary authority that is binding. This helps control legal costs while making legal professionals choosing to use computerized research more efficient. 
         [0008]    However, even with computerized research, legal professionals cannot be fully aware of all potential legal issues due to the vast amount of legal information legal professionals may have to review. For example, West Publishing Company of St. Paul, Minn. (doing business as Thomson West) (hereinafter “West”) offers the ability for legal professionals to conduct computerized research on over 100 million documents. West collects legal content from various sources and makes them available electronically through its Westlaw® information-retrieval system. (Westlaw® is a trademark of West). Searchable documents include documents from both primary and secondary sources. Further, the West Key Number™ System, which provides classified summaries of legal points, made in judicial opinions, is also searchable (West Key Number™ is a trademark of Thomson West). The summaries, known as Headnotes, are classified into more than 90,000 distinct legal categories, and can be used for a variety of purposes, such as evaluating the relevance of legal opinions to particular legal issues. Secondary resources, such as American Law Reports (ALR®), include about 4,000 in-depth scholarly articles, each teaching about a separate legal issue. 
         [0009]    Multiple systems used by legal professionals, including Westlaw®, have addressed the ability of a legal professional performing a search to quickly become familiar with other potentially relevant legal issues. For example, Westlaw® currently provides legal professionals with a feature known as ResultsPlus®. This feature is described in U.S. patent application Ser. No. 11/028,476 filed Jan. 3, 2005, the contents of which are incorporated herein in entirety, entitled “Systems, Methods, Interfaces and Software for Extending Search Results Beyond Initial Query-Defined Boundaries.” Essentially, this feature provides the legal professional with a screen with links to: (1) a first set of documents responsive to their search query; and (2) a second set of documents including documents outside the boundaries of the original search query. The feature does so in a manner that causes the first set of documents and the second set of documents to be displayed separately within a single graphical user interface (“GUI”). 
         [0010]    There exists a need to further improve the searching of legal documents by legal professionals. 
       SUMMARY OF THE INVENTION 
       [0011]    We have recognized that legal professionals are more efficient in conducting legal research if the second set of documents which is outside the boundaries of the original search query is organized into clusters. More specifically, we have invented a method comprising receiving a first signal indicative of a selection of a legal document associated with a set of metadata; based upon the set of metadata, picking a first cluster of legal documents and a second cluster of legal documents, the first cluster of legal documents being associated with a first legal topic and the second cluster of legal documents being associated with a second legal topic; and transmitting a second signal relating to the first cluster of legal documents and the second cluster of legal documents. 
         [0012]    Advantageously, the present invention permits legal professionals to conduct legal research more effectively by providing the legal professional with information regarding clusters of documents associated with a document the legal professional has viewed and/or accessed in some fashion. 
         [0013]    Other advantages will be apparent to those skilled in the art based upon the remainder of the specification, including the figures. 
     
    
     
       BRIEF DESCRIPTION OF THE FIGURES 
         [0014]      FIG. 1  is a diagram of a system corresponding to one embodiment of the invention; 
           [0015]      FIG. 2  is a flowchart corresponding to the operation of the system of  FIG. 1 ; 
           [0016]      FIGS. 3A through 3I  are screenshots of what a user may see when executing methods in accordance with the invention; and 
           [0017]      FIGS. 4A through 4L  relate to the generation of clusters of legal documents. 
       
    
    
     DETAILED DESCRIPTION 
     Background 
       [0018]    In addition to providing this background section, this detailed description will describe a system in which the invention may be implemented, including the system&#39;s components and structure. Next, the detailed description will describe the operation of the system, including a legal professional&#39;s interactions with the system and resulting displays associated with clusters of legal documents. Next, the detailed description will describe how clusters of legal documents are originally generated. Finally, the detailed description will describe how these clusters may be used to provide the user with additional relevant documents or additional relevant clusters. 
         [0019]    As used herein, “topic” and/or “legal topic” shall mean a legal area, issue, and/or subject matter. An example of this is “search and seizure.” A sub-topic and/or “legal sub-topic” shall mean a more granular classification of a topic and/or legal topic. Examples of this are “search and seizure—traffic stop” and “search and seizure—expectation of privacy.” A cluster shall mean a set of documents grouped according to a topic that the documents hold in common. An example of a cluster is a group of legal documents relating to “search and seizure.” Although typically a heterogeneous (containing more than one content type of document) set of documents, it is possible for a cluster to be a homogeneous set of documents (i.e., containing only one content type of document). A noun phrase is a word group that contains a noun and its modifiers. Examples of noun phrases are “product liability action” and “our favorite restaurant.” A segment is a portion of a document that may be defined by the particular topic it addresses. By way of example, a court decision discussing and finding a party liable for fraud and then discussing damages is one document with two segments, namely “fraud” and “damages.” The words pick, choose, select, identify, and all respective forms thereof, shall be used interchangeably. 
         [0020]    Also, a document is “associated” with a cluster if it is relevant to the topic of the cluster. Further, a document is a “member” of a cluster if it is both relevant to the topic associated with a cluster and is important in the context of the topic. Still further, a first document is said to be “similar” to a second document if they share a sufficient number of features such as noun phrases and citation history. 
         [0021]    Finally, it should be noted that there are many different types of legal documents including but not limited to case law, statutes, regulations, administrative decisions, secondary sources, briefs, pleadings, motions, memoranda, expert witness testimony, court orders, scholarly articles, and jury verdicts. Further, these documents arise in the federal, state and/or local context (e.g., a federal court opinion as opposed to a state court opinion). Also, at least some of these types of documents (e.g., non-court decision documents) may be associated with notes of decisions which serve as alerts to the legal professional accessing the documents that the document (or, e.g., the contents of the document such as a statute) has been involved in litigation. Some of these documents are primary authority and some are secondary authority. 
       System Components and Structure 
       [0022]      FIG. 1  shows an exemplary online information-retrieval system  100 . System  100  may include one or more databases  110 , one or more servers  120  (only one shown), and one or more access devices  130  (only one shown). 
         [0023]    Databases  110  includes a set of primary databases  112  and a set of second databases  114 . Primary databases  112 , in the exemplary embodiment, include a case law database  1121  and a statutes databases  1122 , which respectively include judicial opinions and statutes from one or more local, state, federal, and/or international jurisdictions. Secondary databases  114  include an ALR® database  1141 , an AMJUR® database  1142 , a West Key Number™ (KNUM) Classification database  1143 , and a law review (LREV) database  1144 . Other databases (not shown) may include financial, tax, scientific, and/or health-care information. Also, it should be noted that primary and secondary may also connote the order of presentation of search results and not necessarily the authority or credibility of the search results. 
         [0024]    Databases  110 , which take the exemplary form of one or more electronic, magnetic, or optical data-storage devices, include or are otherwise associated with respective indices (not shown). Each of the indices includes terms and phrases in association with corresponding document addresses, identifiers, and other conventional information. Databases  110  are coupled or couplable via a wireless or wireline communications network, such as a local-, wide-, private-, or virtual-private network, to server  120 . 
         [0025]    Server  120  is generally representative of one or more servers for serving data in the form of web pages or other markup language forms. This may be done with known associated applets, ActiveX controls, remote-invocation objects, or other related software and data structures to service clients of various “thicknesses.” More particularly, server  120  includes a processor module  121  and a memory module  122 . 
         [0026]    Processor module  121  includes one or more local or distributed processors, controllers, or virtual machines. In the exemplary embodiment, processor module  121  assumes any convenient or desirable form. 
         [0027]    Memory module  122  takes the exemplary form of one or more electronic, magnetic, or optical data-storage devices. Memory module  122  is comprised of a subscriber database  123 , a search module  124 , a user-interface module  126 , and a cluster module  128 . 
         [0028]    Subscriber database  123  includes subscriber-related data for controlling, administering, and managing pay-as-you-go or subscription-based access of databases  110 . Search module  124  includes one or more search engines and related user-interface components. These search engines receive and process user queries and/or other user activity against one or more of databases  110 , including the primary databases  112  and the secondary databases  114 . The secondary databases may provide, for example, topical treatises, state practice guides, statutes, and/or law review articles to augment searches of case law database. User-interface module  126  includes machine readable and/or executable instruction sets for wholly or partly defining web-based user interfaces, such as search interface  1261  and results interface  1262 , over a communications link  129  such as a wireless or wireline communications network on one or more accesses devices, such as access device  130 . 
         [0029]    Cluster module  128  includes machine readable and/or executable instruction sets. Cluster module  128  interacts, directly and/or indirectly, with the processor  121  and other modules in the memory  122 . Cluster module  128  also interacts, directly and/or indirectly, with the databases  110  via communications links  111  and with access device  130  via communications link  129 . 
         [0030]    Access device  130  is generally representative of one or more access devices, all of which may simultaneously interact with the server  120 . In the exemplary embodiment, access device  130  takes the form of a personal computer, workstation, personal digital assistant, mobile telephone, or any other device capable of providing an effective user interface with a server or database. Specifically, access device  130  includes a processor module  131  one or more processors (or processing circuits)  131 , a memory  132 , a display  133 , a keyboard  134 , and a graphical pointer or selector  135 , such as a “mouse.” 
         [0031]    Processor module  131  includes one or more processors, processing circuits, or controllers. In the exemplary embodiment, processor module  131  takes any convenient or desirable form. Coupled to processor module  131  is memory  132 . 
         [0032]    Memory  132  stores code (machine-readable or executable instructions) for an operating system  136 , a browser  137 , and a GUI  138 . In the exemplary embodiment, operating system  136  takes the form of a version of the Microsoft® Windows® operating system, and browser  137  takes the form of a version of Microsoft® Internet Explorer®. Operating system  136  and browser  137  not only receive inputs from keyboard  134  and selector  135 , but also support rendering of GUI  138  on display  133 . Upon rendering, GUI  138  presents data in association with one or more interactive control features (or user-interface elements). (The exemplary embodiment defines one or more portions of interface  138  using applets or other programmatic objects or structures from server  120 .) 
         [0033]    More specifically, graphical user interface  138  defines or provides one or more display regions, such as a query or search region  1381  and a search-results region  1382 . Query region  1381  is defined in memory and upon rendering includes one or more interactive control features (elements or widgets), such as a query input region  1381 A and a query submission button  1381 B. Search-results region  1382  is also defined in memory and upon rendering includes a first region  1382 A, a second region  1382 B, and a third region  1382 C. Region  1382 A includes one or more interactive control features, such as features A 1 , A 2 , A 3  for accessing or retrieving one or more corresponding search result documents from one or more of databases  110  via server  120 . Region  1382 A, in one embodiment, is the region from which a legal professional may select a legal document. Regions  1382 B and  1382 C are, respectively, regions for displaying information relating to the first cluster of legal documents and the second cluster of legal documents. Such information may include respective titles and/or citations for the corresponding documents. For each such documents and/or cluster, this information may be in the form of a hyperlink or other browser-compatible command input that provides access, ultimately, to the documents and/or cluster of documents via server  120  and databases  110 . 
       System Operation 
       [0034]      FIG. 2  is a flowchart  200  corresponding to operation of the system  100  of  FIG. 1 . Flowchart  200  includes blocks  210  through  270  which are arranged and generally described sequentially. However, those skilled in the art realize that other embodiments of the invention may execute two or more blocks in parallel using multiple processors or processor-like devices or a single processor organized as two or more virtual machines or sub processors. Some embodiments also alter the process sequence or provide different functional partitions to achieve analogous results. For example, some embodiments may alter the client-server allocation of functions, such that functions shown and described on the server side are implemented in whole or in part on the client side, and vice versa. Moreover, still other embodiments implement the blocks as two or more interconnected hardware modules with related control and data signals communicated between and through the modules. Thus, the exemplary flowchart of  FIG. 2  (and elsewhere in this description) applies to software, hardware, and/or firmware implementations. 
         [0035]    The remaining description in the System Operation section refers to  FIGS. 2 through 31  wherein  FIG. 2  outlines the operation of the system  100  and  FIGS. 3A through 3I  are various screenshots as seen from the perspective of a user (e.g., legal professional) using a access device  130  to access the WestlawNext™ online information retrieval system. 
         [0036]    As shown in block  210 , the system  100  generates a signal that ultimately causes a search interface to be presented to a user. The signal is output from server  120  to access device  130  via communications link  129  and stored in memory  132 . GUI  138  provides search region  1381  on the access device  130 . It should be noted that this step assumes that the user operating access device  130  has already successfully logged into the system  100  by supplying an interne-protocol (IP) address for an online information-retrieval system and correct login information (e.g., user identification and password), via the access device  130  and communications link  129 , to the system  100 . An exemplary search interface screen  300  presented to the user is depicted in  FIG. 3A . The search interface  300  includes a query input region  310  in which the user of access device  130  may enter a search query by typing text and submitting the query to system  100 . 
         [0037]    As shown in block  220 , the system  100  receives the query, also known as a search request, and processes the request. To process the request, the server  120  communicates with at least one database from databases  110  and identifies a set of legal documents in response to the search request. Next, the server  120 , via the processor  121  and memory  122 , generates a signal associated with the set of legal documents identified in response to the search request. The signal is transmitted over communications link  129  to access device  130 . The access device  130  displays a screen  320  to the user based upon this signal. Such a screen  320  is depicted in  FIG. 3B . It should be noted that  FIG. 3B  does not contain information (e.g., titles, words describing, hyperlinks to, etc. . . . ) relating to a first cluster of legal documents and a second cluster of legal documents. 
         [0038]    As shown in block  230 , the system  100  receives another signal generated by the user of access device  130  via communications link  129 . This signal is indicative of the user accessing a document from the set of legal documents provided in response to the search request. Accessing may be done in a variety of manners including but not limited to the user: (1) viewing the document on the access device  130 ; (2) printing the document; (3) emailing the document; and (4) setting up an alert with respect to the document. As shown block  240 , the processor  121  and memory  122  begin to process this signal. This is done by identifying a set of metadata associated with the accessed or selected document. This set of metadata is then used to pick a first cluster of legal documents and a second cluster of legal documents as shown in block  250 . The manner in which clusters are picked is by using a pre-computed set of clusters associated with each document. The association process, described in more detail in Cluster Generation section below, uses a combination of similarity measures between the document and/or document metadata and the cluster and/or cluster metadata. These measures include statistics (such as term-frequency and inverse document-frequency) regarding terms, noun phrases, word pairs, text, citations, associated queries, and other items. As shown in step  260 , a signal relating to these clusters is generated and transmitted from server  120  to access device  130  via communications link  129 . Next, the access device  130  displays a screen  330  to the user based upon this signal. Such a screen  330  is depicted in  FIG. 3C . It should be noted that the right hand portion  331  of screen  330  is related to the clusters. It should be noted that the right hand portion  331  of screen  330  is analogous to regions  1382 B and  1382 C of  FIG. 1 . Also, portion  332  of screen  330  is analogous to region  1382 A of  FIG. 1 . 
         [0039]    At this point, the user, who had originally search for “federal arbitration act” (see query input region  310  of  FIG. 3A ), realizes that what is more interesting to the user is a set of documents relating to the topic entitled “Alternative Dispute Resolution” (see generally the right hand portion  331  of  FIG. 3C  showing multiple clusters). More specifically, the user wants more information on the sub-topic entitled “Interstate Commerce Requirement of [the] Federal Arbitration Act.” When the users clicks on the appropriate hyperlink relating to the sub-topic, a signal is sent from the access device  130  to the server  120  via the communications link  129 . As shown in block  270 , the server  120  receives and processes this signal by identifying legal documents associated with the topic and sub-topic “Alternative Dispute Resolution/Interstate Commerce Requirement of [the] Federal Arbitration Act.” To process the signal, the server  120  communicates with at least one database from databases  110  and identifies legal documents relevant to the sub-topic (based upon clusters and “sub-clusters”). Next, the server  120 , via the processor  121  and memory  122 , generates a signal associated with the legal documents and transmits it over communications link  129  to access device  130 . The access device  130  displays a screen  340  to the user. Such a screen  340  is depicted in  FIG. 3D . 
         [0040]      FIGS. 3E through 3I  show another series of screen shots relating to the invention. Essentially they illustrate that another scenario under which signals relating to multiple clusters may be transmitted to an access device. It does not have to be initiated solely in response to a “word” or “text” search (as shown in input region  310  of  FIG. 3A ). For example,  FIG. 3E  begins with a user searching for a document associated with a particular citation, namely 489 U.S. 468, a citation to a Supreme Court case. 
       Cluster Generation 
       [0041]      FIGS. 4A through 4J  disclose various algorithms, features and applications for generating and using clusters of legal documents. As discussed in detail below, in one embodiment, the cluster module  128  of  FIG. 1  defines and generates a cluster by identifying one or more legal issues among case-law documents, populates the cluster with a rich spectrum of legal documents based upon the cluster&#39;s legal issue, summarizes the content represented by the generated cluster, and provides various associations between generated clusters and documents, queries, and folders. Although the description below refers to a Westlaw® system environment, one skilled in the art will appreciate that the disclosed algorithms, features and applications are applicable to other online legal research systems. 
         [0042]    To identify one or more legal issues among case-law documents, the cluster module  128  implements a bottom-up strategy. For example, in one embodiment, the cluster module  128  identifies the legal issues inside one document, and then merges similar issues together to form clusters for all documents. 
         [0043]    The cluster module  128  identifies legal issues using a Headnotes grouping defined for a case. For example, for cases deemed important on the Westlaw® system, Headnotes (e.g., editorial annotations) are added during the publishing process. Headnotes provide a succinct summary of a legal issue raised in the case and are also associated with one or more Westlaw® Key Numbers™, described below. An example of a Headnotes grouping with Key Numbers™ is shown in  FIG. 4A . 
         [0044]    Advantageously, by grouping Headnotes based on their “similarities”, the cluster module  128  identifies major legal issues inside a case. In one embodiment, to determine similarity, the cluster module  128  first computes several features from the Headnotes and then applies an agglomerative clustering algorithm. Exemplary similarity features computed by the cluster module  128  include a Key Numbers™ similarity feature, a Headnote text similarity feature, a KeyCite® similarity feature, and a Common Noun Phrase frequency feature. 
         [0045]    The Key Numbers™ similarity feature is based on a Key Number™. West&#39;s Key Number System® is a taxonomy defined on the Westlaw® system that categorizes legal topics into a hierarchical structure. The cluster module  128  computes the similarity between Key Numbers™ based on the global co-existence of Key Numbers™ inside cases. In one embodiment, the cluster module  128  determines Key Number™ topic commonality. 
         [0046]    The Headnote text similarity feature is based on text describing a legal issue. For example, in the Westlaw® system, each Headnote typically includes an amount of text describing a legal issue. The cluster module  128  computes the similarity between two Headnotes&#39; text using wordpair features extracted from them. In one embodiment, the cluster module  128  uses a hybrid approach which combines the TF-IDFs (term-frequency-inverse document-frequency) and probabilities of wordpairs. 
         [0047]    The KeyCite® similarity feature is based on relationships between cases. In the Westlaw® system, KeyCite® data maintains citing and cited relationships between cases (several down to the Headnotes level). In addition, KeyCite® data includes information concerning the importance/authoritativeness of a case, and information regarding similarity among Headnotes (for example, if two or more Headnotes are co-cited together in several cases, they tend to discuss closely related legal issues). U.S. Pat. No. 7,529,756 issued on May 5, 2009 entitled “System and Method for Processing Formatted Text Documents in a Database” (filed Nov. 22, 2000 and assigned U.S. patent application Ser. No. 09/746,557) and U.S. patent application Ser. No. 12/432,380 entitled “System and Method for Processing Formatted Text Documents in a Database” filed on Apr. 29, 2009 describe KeyCite® in detail and are both incorporated in their entirety. The cluster module  128  computes the frequency of how often Headnotes have been co-cited in other cases. 
         [0048]    The Common Noun Phrase frequency feature is based on a noun phrase (NP) whose head is a noun or a pronoun, optionally accompanied by a set of modifiers. In the Westlaw® system, NPs typically represent a legal term in a Headnote. The cluster module  128  computes the frequency of two common NPs between Headnotes, which provides a measure of how similar Headnotes are at the “concept” level. In one embodiment, the cluster module  128  uses the NP frequency feature as a supplement to the Headnote text similarity features, since a NP may be considered an n-gram for a particular value of n. 
         [0049]    Once the cluster model  128  computes one or more similarity features between Headnotes, the cluster module  128  implements an agglomerative clustering algorithm to group similar Headnotes. For example, in one embodiment, the cluster module  128  merges two Headnotes together while maximizing the following equations, 
         [0000]    
       
         
           
             
               H 
               2 
             
             = 
             
               maximize 
                
               
                   
               
                
               
                 
                   τ 
                   2 
                 
                 
                   ɛ 
                   1 
                 
               
             
           
         
       
       
         
           
             where 
             , 
             
               
 
             
              
             
               
                 τ 
                 2 
               
               = 
               
                 maximize 
                  
                 
                   
                     ∑ 
                     
                       r 
                       = 
                       1 
                     
                     k 
                   
                    
                   
                     
                       ∑ 
                       
                         
                           h 
                           i 
                         
                         ∈ 
                         
                           S 
                           r 
                         
                       
                       
                           
                       
                     
                      
                     
                       cos 
                        
                       
                         ( 
                         
                           
                             h 
                             i 
                           
                           , 
                           
                             C 
                             r 
                           
                         
                         ) 
                       
                     
                   
                 
               
             
           
         
       
       
         
           
             
               ɛ 
               1 
             
             = 
             
               minimize 
                
               
                 
                   ∑ 
                   
                     r 
                     = 
                     1 
                   
                   k 
                 
                  
                 
                   
                     n 
                     r 
                   
                    
                   
                     cos 
                      
                     
                       ( 
                       
                         
                           C 
                           r 
                         
                         , 
                         C 
                       
                       ) 
                     
                   
                 
               
             
           
         
       
       
         
           
             
               C 
               r 
             
             = 
             
               
                 
                   ∑ 
                   
                     h 
                     ∈ 
                     
                       S 
                       r 
                     
                   
                 
                  
                 h 
               
               
                 n 
                 r 
               
             
           
         
       
       
         
           
             C 
             = 
             
               
                 
                   ∑ 
                   
                     
                       S 
                       r 
                     
                     ∈ 
                     S 
                   
                 
                  
                 
                   ( 
                   
                     
                       
                         ∑ 
                         
                           h 
                           ∈ 
                           
                             S 
                             r 
                           
                         
                       
                        
                       h 
                     
                     
                        
                       
                         S 
                         r 
                       
                        
                     
                   
                   ) 
                 
               
               k 
             
           
         
       
     
         [0000]    in which τ 2  is the intra-cluster similarity and ε 1  is the inter-cluster similarity. In these equations, k being the total number of clusters, S r  being one of the k clusters, and S being the collection of all the clusters,
 
h i  being one of the Headnote in the cluster S r  
 
C r  being the center of one cluster,
 
C being the center of all the clusters
 
n r  being the number of Headnotes in the cluster S r .
 
In one embodiment, the cluster module  128  scans through all the Headnote feature vectors, which is one common representation for a set of features used, and identifies two feature vectors which have the maximal τ 2  value. The cluster module  128  also computes the value ε 1  at approximately the same time. The cluster module  128  stops the scanning iteration when the value ε 1  is less than a predefined threshold. The cluster module  128  stops the scanning iteration when the value of ε 1  is less than a predefined threshold. The range of the threshold is between 0.0 to 1.0, and preferable, it is set to be 0.45.
 
         [0050]    Advantageously, by utilizing a predefined threshold, the cluster module  128  avoids setting up the number of clusters for the data set in advance, which many of the known clustering algorithms require. The cluster module  128  applies this technique to cases with Headnotes and resulting topics are used in a cluster merging process described below which produces clusters for cases. 
         [0051]    Once topics are determined, the cluster module  128  is configured to merge similar clusters. For example, legal topics detected in different cases using the before-mentioned techniques may be very similar, i.e., they are concerned with the same or closely related legal issues. By merging similar clusters together, the cluster module  128  partitions the legal space into meaningful clusters. 
         [0052]    In one embodiment, the cluster module  128  mergers clusters using a two step process. First, the cluster module  128  performs a candidate selection process. 
         [0053]    The candidate selection process includes generating, training and applying three different CaRE® indices to eligible topics. CaRE® stands for Classification and Recommendation Engine. CaRE® is described in detail in U.S. Pat. No. 7,062,498 which issued on Jun. 13, 2006 entitled “Systems, Methods, and Software for Classifying Text from Judicial Opinions and other Documents” (filed on Dec. 21, 2001 and assigned U.S. patent application Ser. No. 10/027,914), U.S. Pat. No. 7,580,939 which issued on Aug. 25, 2009 entitled “Systems, Methods, and Software for Classifying Text from Judicial Opinions and other Documents” (filed on Aug. 30, 2005 and assigned U.S. patent application Ser. No. 11/215,715), and U.S. patent application Ser. No. 12/545,642 entitled “Systems, Methods, and Software for Classifying Text from Judicial Opinions and other Documents” filed on Aug. 21, 2009, all of which are incorporated in their entirety. 
         [0054]    In one embodiment, the cluster module  128  performs the following indexing functions: CaRE® word-pairing indexing, CaRE Key Numbers™ indexing, and CaRE® citation indexing. 
         [0055]    In CaRE® word-pair indexing, the cluster module  128  associates each topic with a number of Headnote texts. The cluster module  128  computes word-pairs of the text and indexes them. The cluster module  128  retrieves a list of topics based on the similarities between word-pair profiles. 
         [0056]    In CaRE Key Numbers™ indexing, the cluster module  128  associates each topic with a list of Key Numbers™ via Headnotes. The cluster module  128  then computes indexed Key Number™ profiles. The cluster module  128  then retrieves a list of topics based on the commonalities between Key Number™ profiles. 
         [0057]    In CaRE® citation indexing, the cluster module  128  links each topic to one or more cases, each case is further linked to other cases via KeyCite® information (contain both citing and cited information). The cluster module  128  also computes citation profiles that are indexed. The cluster module then retrieves a list of topics based on common citation patterns between citation profiles. 
         [0058]    Advantageously, by aggregating the recommendations from the three generated CaRE® indices, the cluster module  128  generates a list of candidates for each of the topics. 
         [0059]    Second, from the list of candidates generated from the selection process, the cluster module  128  determines for each cluster whether the cluster is “similar” to an input topic, and thus should merged with the topic. In one embodiment, for each topic identified, the cluster module  128  generates a query during the Headnotes grouping phrase described previously. The query can include noun phrases and Key Numbers™. An example is shown in connection with  FIG. 4B . From the query, along with the associated cases, the cluster module  128  determines several features. Exemplary features calculated by the cluster module  128  include Noun Phrases (NPs) similarity—which includes a global maximal score between pair-wise NPs, mean of maximal score between pair-wise NPs, percentage of common NPs, and percentage of common words, Key Numbers™ (KNs) similarity—which includes a Key Number™ profiles similarity score, percentage of common KNs, and percentage of common KN topics, Co-citation feature—which describes the normalized number of documents cited by both associated seed cases, and Co-click feature, which calculates the normalized number of sessions that have both associated seed cases. The Co-citation feature describes the normalized number of documents cited by both associated seed cases, and is computed using the following formula: 
         [0000]    
       
         
           
             
               cite_sim 
                
               
                 ( 
                 
                   
                     c 
                     i 
                   
                   , 
                   
                     c 
                     j 
                   
                 
                 ) 
               
             
             = 
             
               
                 cite 
                  
                 
                   ( 
                   
                     
                       c 
                       i 
                     
                     ⋂ 
                     
                       c 
                       j 
                     
                   
                   ) 
                 
               
               
                 cite 
                  
                 
                   ( 
                   
                     
                       c 
                       i 
                     
                     ⋃ 
                     
                       c 
                       j 
                     
                   
                   ) 
                 
               
             
           
         
       
     
         [0000]    in which cite(c i ∩c j ) is the count of other legal documents citing both seed cases ci and cj. Also, cite(c i ∪c j ) is the count of legal documents citing either seed cases ci or cj.
 
The co-click feature calculates the normalized number of sessions that have both associated seed cases and is be computed using the following formula:
 
         [0000]    
       
         
           
             
               coclick_sim 
                
               
                 ( 
                 
                   
                     c 
                     i 
                   
                   , 
                   
                     c 
                     j 
                   
                 
                 ) 
               
             
             = 
             
               
                 click 
                  
                 
                   ( 
                   
                     
                       c 
                       i 
                     
                     ⋂ 
                     
                       c 
                       j 
                     
                   
                   ) 
                 
               
               
                 click 
                  
                 
                   ( 
                   
                     
                       c 
                       i 
                     
                     ⋃ 
                     
                       c 
                       j 
                     
                   
                   ) 
                 
               
             
           
         
       
     
         [0000]    in which click(c i ∩c j ) is the count of sessions in which both seed cases ci and cj were clicked. Also, click(c i ∪c j ) is the count of sessions in which either seed cases ci or cj were clicked. 
         [0060]    In one embodiment, the cluster module  128  uses these generated features to train a support vector machine (“SVM”) ranker model. SVMs and ranking is well known in the art. In order to provide target data for the training of the model, the cluster module  128  generates a set of “silver” preference grades automatically that measure overlaps between recommended cases from the queries through a search engine process. In order to provide target data for the training of the model, the cluster module  128  generates a set of “silver” preference grades automatically by measuring the overlaps between retrieved cases using the queries associated with the clusters through a search engine process. The search engine is described in detail in U.S. patent application Ser. No. 11/538,749 filed Oct. 4, 2006 entitled “Systems, Methods, and Software for Identifying Relevant Legal Documents” (now Publication No. U.S. 2008/0033929 A1). By ranking the scores of the candidates using the features via the SVM model, the cluster model  128  generates a cluster by merging selected candidates with a seeding topic based on the ranked scores. A list of clusters can then be produced by exhaustively repeating this process for each of the topics such that one is either merged with other topics or becomes a seeding topic. 
         [0061]    Once the list of clusters is selected to be merged, the cluster module  128  generates labels. A label displays the “aboutness” of a cluster and reflects a summary of the content inside the cluster. The content of a populated cluster can include cases, statutes, regulations, administrative decisions, analytic materials, briefs, expert witness testimony, jury verdict reports, state trial court orders, pleadings, motions and memoranda as well as other legal documents. Furthermore, the cases and some of the other documents will also include Headnote texts and Key Numbers™. The catchline of a Key Number™ is a short description of a defined legal topic, and it is hierarchically structured such that the first portion is often referred to as the Key Number™ topic, such as “Negligence” in  FIG. 4C , and subsequent portions are often referred to as Key Number™ sub-topics, while the last portion is often referred to as the leaf level. 
         [0062]    In one embodiment, the cluster module  128  generates a hierarchical label structure that includes a topic, optional sub-topic, and a noun phrase from cases. The topic and sub-topic parts are derived from Key Number™ catchlines, which are precise and hierarchically structured phrases describing various legal issues. The noun phrase is selected from Headnote texts inside a cluster. Examples of a cluster label is shown below wherein the bold portions represent the topic and sub-topic, and the italic portion is the NP. 
         [0063]    Securities regulation/state regulation/investment contract security 
         [0064]    Schools/Teachers&#39; Duties and Liabilities/Governmental Immunity 
         [0065]    Typically, a cluster contains a certain number of Key Numbers™, typically those assigned to the Headnotes contained in the cluster. To generate the topic and sub-topic portion of the label, the cluster module  128  computes a frequency of the Key Numbers™ which results in major topics included in the cluster being determined. Once a major topic has been identified, the cluster module  128  traverses the catchlines among Key Numbers™ in the major topic to determine a sub-topic. In one embodiment, the cluster module  128  traverses the catchlines until a divergence is detected based on a majority voting scheme. An example of label generation for topics is shown in connection with  FIG. 4C  wherein the label is shown in box  410 . An example of a majority voting scheme is one where the top n post-divergence sub-topics are considered (where n might be, for example 7) and which selects the sub-topic that is the most frequently occurring within the candidate set. 
         [0066]    The cluster module  128  generates the noun phrase portion by extracting all the Headnote texts inside a cluster. In one embodiment, only those Headnotes in the major Key Number™ topics are selected by the cluster module  128  for this process. Several features are derived for each of the noun phrases, and the top scored noun phrase (NP) is selected by the cluster module  128  as part of the label. For example, in one embodiment, the several features include the length of the NP, the term frequency of the composite NP, the term frequency of the NP&#39;s terms considered jointly, and the TF-IDF score using normalized TF, As used above, TF stands for term frequency within the given document, DF stands for document frequency within the given collection, and IDF stands for inverse document frequency or the reciprocal of the document frequency. In the given embodiment, weights are determined for this set of features so as to optimize the performance of the label selection process based on empirical evidence from a label grading process. It is also worth noting that for NP scoring and selection purposes an NLP simplified version of the extracted NPs are used (stopped, stemmed, etc.), By contrast, for presentation purposes, a canonical (original) form of the NP is used for user readability. 
         [0067]    Because of the importance of each cluster possessing a label that is unique across the set of clusters, two types of uniqueness (or duplication) checks are performed. In order to apply these checking processes to the entire cluster set, the clusters are first ranked by a fitness function that relies on many factors including but not limited to the number of initial cases in the given cluster, and additional features such as the popularity of the cases in the cluster (based on citations and based on user selection), the number of jurisdictions represented, the average age of the cases in the cluster, and the average age of the Key Numbers™ in the cluster. Such a fitness function effectively enables one to rank the clusters by a quality metric. 
         [0068]    Once the clusters are ranked according to the fitness function, the labeling process is applied to the highest quality cluster first, then the next highest, etc. . . . At the same time, the resulting labels are recorded and if a given label has already been assigned to a previously processed cluster, the candidate label is rejected in favor of the next candidate label that has not been previously assigned. Similarly, a semantic representation of each label is recorded, and each candidate label is also assessed for its semantic uniqueness. If a highly semantically similar label has already been assigned, a label can be rejected for a less semantically similar label. Processing for this semantic comparison process includes basic natural language processing such as stopping, stemming, term deduping, etc. A threshold may also be invoked such that if the core constituent tokens in two labels being compared are 80% similar, they are considered semantically similar, and the candidate will be rejected in favor of the next candidate that is not found to be semantically similar using this threshold. 
         [0069]    Once all of the clusters are identified and defined in the legal space by the cluster module  128 , various legal documents are associated with the predefined legal clusters. For example, when a legal document is presented for display in an online legal research system, such as Westlaw®, all the legal topics discussed in the document are automatically identified and associated with related clusters, which can relay all related cases, statutes, regulations, and other documents that discuss the same legal issues as in the original document. To relay all the related documents, in one embodiment, the cluster module  128  applies the search engine process as described in Publication No. U.S. 2008/0033929 A1 using the generated query of a cluster. The query of a cluster comprises of a number of noun phrases and key numbers. The selection of the noun phrases and key numbers are based on their importance to the defined legal topics using the similar features as in the labeling process. By adding key numbers into the query of a cluster, the cluster module  128  can tailor the search engine to retrieve the most relevant cases, statutes, regulations, and other documents either online (in real-time) or offline (pre-population). 
         [0070]    An example workflow of document cluster association is shown in connection with  FIG. 4D . As shown in  FIG. 4D , in one embodiment, for an incoming document, a list of legal topics described in the document is determined by the cluster module  128 . For each topic, a list of similar clusters is associated and recommended. 
         [0071]    Depending on the metadata available, four different techniques are implemented by the cluster module  128 , as illustrated in  FIG. 4E . For documents with Headnotes defined (cases, some administrative decisions and briefs), the cluster module  128  process operates similarly to the process described in connection with finding legal issues via Headnotes grouping discussed above. 
         [0072]    For some statutes and regulations that include attached notes of decisions (NODs)—a compilation of cases that construe or apply the Statutes or Regulations, the NODs are detailed into the Headnote level for each of the case. As such, the cluster module  128  identifies Key Number™ information from them. The cluster module  128  then groups these Key Numbers™ based on their catchlines such that Key Numbers™ with the most common sub-topics are grouped into one group. An example of Key Numbers™ grouping is shown in connection with  FIG. 4F . 
         [0073]    As shown in the  FIG. 4F  example, five (5) Key Numbers™ are shown and after grouping by the cluster module  128 , three Key Numbers™ 197K201, 197K202, and 197K203 are grouped into one group since they have common sub-topics up to 1971(A)1 (“Nature of Remedy in General”), and two Key Numbers™ 197K912, 197K913 are grouped in another group for the sub-topic 197V (“Suspension of writ”). These grouped Key Numbers™ define the topics of the document. 
         [0074]    For documents with citing documents and no Headnotes or NODs, the cluster module  128  incorporates two pieces of information into its method: one is from all the Key Numbers™ of the cited cases and another is from Key Numbers™ suggested by CaRE-KNA™ using the document text. KNA stands for Key Number™ Assignments. The cluster module  128  groups these two sets of Key Number™ by their topics and then sorts them based on topic popularity. The Key Numbers™ from the cases side with the highest topic popularity that agree with the Key Numbers™ from the CaRE-KNA™ side describing the topic level are selected by the cluster module  128  to generate legal topics for the document. Groupings, similar to those shown in  FIG. 4F , are made by the cluster module  128 . One grouping is from all the Key Numbers™ of the cited cases and another is from Key Numbers™ suggested by CaRE-KNA™ service using the summarized document text. In one embodiment, the summarized document text comprises the first 2,000 characters of the document. Those skilled in the art will realize there are other methods for generating summaries of legal documents. Examples of such methods may be found in Schilder, F. and Kondadadi, R,  FastSum: Fast and accurate query - based multi - document summarization  as contained in the proceedings of the Joint Annual Meeting of the Association for Computational Linguistics and the Human Language Technology Conference (ACL-HLt 2008), pages 205-208, Columbus, Ohio, June 2008. CaRE-KNA™ is a Key Number Assignment™ service built upon the CaRE® indexing system using the collection of the Key Numbers™ with their corresponding Headnote texts. It can recommend the most relevant Key Numbers™ based on an input query text. The cluster module  128  groups these two sets of Key Numbers™. 
         [0075]    For documents with no meta-data but text, the cluster module  128  applies a CaRE-KNA® service to suggest Key Numbers™ based on the text. The Key Numbers™ with the highest topic popularity are then used by the cluster module  128  to perform tasks similar to those shown in  FIG. 4F  to generate legal topics for the document. 
         [0076]    After each legal topic has been identified by the cluster module  128  for a document, the cluster module  128  associates each document with the pre-defined legal clusters based on its similarity. 
         [0077]    For example, in one embodiment, the association candidate selection process executed by the cluster module  128  is similar to the candidate selection process for merging clusters described previously. In particular, for all the clusters which can be associated to the topics in legal documents, the cluster module  128  generates the three CaRE® indices based on the word-pair features, Key Number™ profiles features, and KeyCite® citing/cited profiles features. 
         [0078]    For each topic, the three sets of features described previously, namely the word-pair features based on the Headnote text, the Key Number™ profiles features, and the KeyCite® citing/cited profiles of the seeding document, are calculated by the cluster module  128  and sent to the CaRE® indices. Each CaRE® engine is used to retrieve its independent suggestions, which aggregated later to form a list of candidates to be associated.  FIG. 4G  shows an example flowchart of the association candidate selection process. Next, the cluster module  128  computes a list of features, as shown and described in connection with  FIG. 4H . Next, the cluster module  128  applies a SVM ranker to these computed features. The cluster module  128  then selects the top scored candidates as the associated clusters to the topics.  FIG. 4I  shows a flowchart illustrating this process. 
         [0079]    In one embodiment, the cluster module  128  associates sets of documents stored in folders with a set of document recommendations that address the same legal issue(s) which are relevant to the original document set. For example, in the Westlaw® system, a “Research Folder” is a place where a user can store together one&#39;s documents of interest. The research folder can contain various numbers of documents and various document types. This folder-based document recommendation method executed by the cluster module  128  identifies common topics (legal issues) among these foldered documents and proceeds to return additional relevant documents that discuss the same topics. 
         [0080]    For example, in one embodiment, input to the method is a list of documents, such as cases, statutes, and regulations found in a folder box  480  of  FIG. 4L . The output of the method is a list of additional documents addressing the same distinct legal issues. The method involves two steps. The cluster module  128  first detects topics and then retrieves the additional documents which share the same legal issues as shown in functional box  481  of  FIG. 4L . 
         [0081]    In the topic similarity detection step, which takes place in functional box  482 , the cluster module  128  uses the relationships among the documents in a folder to find additional relevant documents. Advantageously, instead of utilizing the document content itself, which may be computationally expensive, these relationships identified in the document metadata are exploited. 
         [0082]    These document relationships are quantified by a similarity matrix based on two sources of information. One is the cluster memberships of the documents in the folders. The second is the citation information associated with the documents, citing as well as cited citation information. 
         [0083]    The dimension of the similarity matrix is n×n, where n is, for example, the number of legal cases. Such a matrix could also include other document types such as legal briefs, for example. Each entry of the matrix, a ij , is the similarity score of the document in row i and the document in column j. In the typical embodiment, the matrix is sparse (that is, the majority of the entries have 0 values). This property allows for an efficient storage of the entries in a database. 
         [0084]    The matrix is computed offline and the results (entries) are stored in a database (which is part of  482  of  FIG. 4L . On the online side, an item-based top-N ranking algorithm is used in functional blocks  483  and  484  to “recommend” the top N documents in response to the documents stored in folders  480 . 
         [0085]    In practice, one matrix is generated based on document cluster memberships and another matrix is generated based on document citation information. Since both cluster membership and citation information can be used, there can be two scores that exist between two documents. 
         [0086]    Once these matrices have been generated offline, on the online side, the inputs of the recommendation algorithm are the document identifiers of the source documents (from the given folder) along with other useful metadata, for example, the jurisdictions of the documents. In one embodiment, where two sources of information are used, two recommendation algorithms are in fact run, one based on membership, the other based on citation information, ranking results from both and then combining these results. Below is a set of pseudo-code for the recommendation algorithm based upon cluster memberships. 
         [0000]    
       
         
               
             
               
             
               
               
               
             
               
             
           
               
                   
               
               
                 Recommend docs based on memberships 
               
               
                   
               
             
             
               
                   
               
             
          
           
               
                 Input: f 1 ,f 2 ,f n , jurisdiction (f i , is a case in the target folder i), n&gt;2 
               
               
                 Steps: 
               
             
          
           
               
                   
                 1. 
                 Get d k  (the kth document) for each s i , (the similarity score) where 
               
               
                   
                   
                 sim1(s i ,s j )&gt;=0.05 in the same jurisdiction 
               
               
                   
                   
                 If sim1(d k ,s i ,)&gt;0.05, count(d k )++ 
               
               
                   
                 2. 
                 Compute w(s i ), which is the pagerank of s i , based on the graph  
               
               
                   
                   
                 with source cases only 
               
               
                   
                 3.  
                 Compute score(d k ), if count(d k ) is not less than T (an empirical  
               
               
                   
                   
                 threshold of 500)T = max(M, M/2+2), M is the number of s i , 
               
               
                   
                   
                 whose w(s i )&gt;0 
               
               
                   
               
               
                   
                   
                 
                   
                     
                       
                         
                           score 
                            
                           
                             ( 
                             
                               d 
                               k 
                             
                             ) 
                           
                         
                         = 
                         
                           
                             count 
                              
                             
                               ( 
                               
                                 d 
                                 k 
                               
                               ) 
                             
                           
                           × 
                           
                             
                               ∑ 
                               
                                 s 
                                 i 
                               
                               
                                   
                               
                             
                              
                             
                                 
                             
                              
                             
                               ( 
                               
                                 
                                   sim1 
                                    
                                   
                                     ( 
                                     
                                       
                                         d 
                                         k 
                                       
                                       , 
                                       
                                         s 
                                         i 
                                       
                                     
                                     ) 
                                   
                                 
                                 × 
                                 
                                   w 
                                    
                                   
                                     ( 
                                     
                                       s 
                                       i 
                                     
                                     ) 
                                   
                                 
                               
                               ) 
                             
                           
                         
                       
                     
                   
                 
               
               
                   
               
               
                   
                 4. 
                 Rank d k  based on the score from step 3 
               
             
          
           
               
                 Output: a list of top k cases (k&lt;=10)with its ranking 
               
               
                   
               
             
          
         
       
     
         [0087]    In another embodiment, one could make recommendations in response to documents in a folder that are in fact clusters rather than documents. Such an embodiment may aggregate clusters among those associated to the documents in the folders, and may assign each cluster a combined score (defined as S COMB ). The cluster module  128  then sorts these scores in descending order. The cluster module  128  computes the combined score S COMB  based on topic scores S TP  (rank can be implied as well, defined as R TP ), cluster scores S AC  (rank can be implied as well, defined as R AC ), and the frequency count (defined as f). 
         [0088]    For example, referring to Fig. J, one folder includes three documents (Doc  1   450 , Doc  2   460 , and Doc  3   470 ). The first document (Doc  1   450 ) includes two topics (Topic  1   451  and Topic  2   452 ), and the second and third documents include one topic (Topic  1   461 ). Further, each topic is associated with two clusters (Clu N). The combined score S COMB  computed for Clu  1  is a combination of frequency count (f=2 since the Clu  1  is associated in two topics), scores from topics S TP  (two scores, one from Doc  1 -&gt;Topic  1 , and another from Doc  2 -&gt;Topic  1 ), ranks from topics R TP  (rank is inferred by the scores and normalized based on a power based function, clusters from lower ranked topics use a lower weight), scores from association S AC , and ranks from association R AC  (similar normalization is applied). The cluster module  128  computes the combined scores S COMB  for other clusters in a similar fashion. In one embodiment, the cluster module implements the following formula to compute the combined score S COMB  for each of the clusters, 
         [0000]    
       
         
           
             
               S 
               COMB 
             
             = 
             
               
                 ∑ 
                 
                   i 
                   = 
                   1 
                 
                 f 
               
                
               
                 ( 
                 
                   
                     S 
                     
                       TP 
                       i 
                     
                   
                   × 
                   
                     B 
                     
                       ( 
                       
                         
                           R 
                           
                             TP 
                             i 
                           
                         
                         - 
                         1 
                       
                       ) 
                     
                   
                   × 
                   
                     S 
                     
                       AC 
                       i 
                     
                   
                   × 
                   
                     B 
                     
                       ( 
                       
                         
                           R 
                           
                             AC 
                             i 
                           
                         
                         - 
                         1 
                       
                       ) 
                     
                   
                 
                 ) 
               
             
           
         
       
     
         [0000]    where B is a constant, i is the ith cluster, and R x =1, 2, . . . . Preferably, B is equal to 0.9. An example of output generated by the cluster module  128  after topic detection is shown in Fig. J. 
         [0089]    In the topic consolidation step, the cluster module  128  condenses the aggregated clusters into groups such that each group contains highly “similar” clusters, and a representative cluster is selected to for each of the group. 
         [0090]    For example, in one embodiment, the cluster module  128  scans through the ordered clusters list and performs a pair-wise similarity comparison between clusters using the information extracted from their queries, namely the NPs and the Key Numbers™. For clusters with similarity scores above a certain thresholds, the cluster module  128  merges those clusters into a single group. In one implementation using a range of similarity scores from 1 through 5 (with 5 being the most similar), the threshold is 2.7. The cluster module  128  then selects the cluster ranked highest in the ordered list (from the topic detection step described previously with reference, in part, to  FIG. 4J ) to be the representative of the group. The cluster module  128  computes the score of the selected group as the sum of the scores of the clusters in the group. The remainder of the clusters in the group are not visible as output of the algorithm. After the comparison is complete, the cluster module  128  sorts the cluster groups by group score in descending order. 
         [0091]    Example output of the topic consolidation step is shown on the right side of  FIG. 4K . As shown in  FIG. 4K , in the output, the cluster module  128  grouped clusters Clu  1 , Clu 2  and Clu 5  together, as these clusters were determined to be similar. Also, clusters Clu  3  and Clu  4  were determined to be similar and thus grouped together. The cluster module  128  then uses the Clu  1  cluster as being representative of the first group, and cluster Clu  3  as being representative of the second group. Clusters Clu  2 , Clu 5  and Clu 4  are not made available in the output. 
         [0092]    The cluster module  128  also provides a query to clusters association. The method of query to cluster association used by the cluster module  128  is similar to the process described in connection with documents having no meta-data. In this case, the query is considered the text. Reference may be made to  FIG. 4F  and associated description for a more detailed explanation. 
         [0093]    While the various sections of the detailed description above are intended to illustrate and teach ways of practicing the current invention, those skilled in the art will appreciate that the invention is not limited to the detailed description. For example, the invention may be used in other information solutions environments relating to, e.g., financial information, health information, tax and accounting information, scientific information and/or combinations of the same. Thus, the scope of the invention is defined by the claims below and their equivalents.