Abstract:
A search engine is disclosed that utilizes both record based data and user activity data to develop, update and refine ranking protocols and to identify words and phrases that give rise to search ambiguity so that the engine can interact with the user to better respond to user queries and enhance data acquisition from databases, intranets and internets.

Description:
RELATED APPLICATION  
       [0001]    This application claims priority to United States Provisional Application Ser. No. 60/127,441 filed Apr. 1, 1999, incorporated herein by reference. 
     
    
     
       BACKGROUND OF THE INVENTION  
         [0002]    1. Field of the Invention  
           [0003]    The present invention relates to a search engine that captures and stores user activity data and uses the stored data to refine search rankings and to methods implemented on a computer for capturing, storing and analyzing user activity data and refining rankings in subsequent searches.  
           [0004]    More particularly, the present invention relates to a search engine that after displaying search results captures and stores user activity data. The user activity data is then analyzed and used to refine ranking in search results in subsequent searches where the ranking protocol will include user activity data, record based data and relationships derived therefrom. The accumulated user activity data can be used to improve search results, improve search efficiency and improve list rankings so better service the user.  
           [0005]    2. Description of the Related Art  
           [0006]    Search engines are well-established in the industry. The search engines provide users with a procedure for finding data in a database. Unfortunately, the current search engines use a ranking for displaying records that meet a particular query in whole or in part which is derived from the data in the record that meet the query criteria. Thus, if a record includes many references to a given search term, it may be placed higher on the list which the user must scroll through. There simply is no user derived data other than the search query to aid in ranking records. Thus, there is a need in the art for a search engine that incorporates into its ranking criteria the user activity data.  
         ABBREVIATIONS AND DEFINITIONS  
         [0007]    The following words and/or abbreviations which may appear throughout this disclosure will have the meaning set forth below:  
           [0008]    URL means universal record location.  
           [0009]    UAD means user activity data.  
           [0010]    RSD means record specific data.  
           [0011]    RP means ranking protocol.  
           [0012]    DB means database.  
           [0013]    URLDB means universal record location database.  
           [0014]    DPU means a digital processing unit.  
           [0015]    CPU means a central processing unit.  
           [0016]    Token means a character string (keyword), punctuation and operators.  
           [0017]    Keyword means a unique character string which appears in a database entry.  
           [0018]    Operator means any Boolean operator for connecting keywords.  
           [0019]    Query means a collection of keywords, punctuation and operators that represents a request for data from a user.  
         SUMMARY OF THE INVENTION  
         [0020]    Devices and Software Implemented in a DPU  
           [0021]    The present invention provides a search engine component implemented in a DPU including:  
           [0022]    a packet processing module that converts each packet to at least one transaction and processes the transaction;  
           [0023]    a database including records, keywords, UAD, and a UAD updating routine for updating UAD; and  
           [0024]    a ranking protocol including RSD and UAD for ranking retrieved records from the records database.  
           [0025]    The present invention provides a search engine component including:  
           [0026]    a packet processing module that converts each packet to at least one transaction and processes the transaction;  
           [0027]    a database module including:  
           [0028]    at least one records database; that may include UAD  
           [0029]    a keywords database including UAD; and  
           [0030]    a UAD updating routine for updating UAD; and  
           [0031]    a ranking protocol including RSD and UAD for ranking retrieved records from the records database.  
           [0032]    The present invention provides a search engine component including:  
           [0033]    a packet processing module that converts each packet to at least one transaction and processes the transaction;  
           [0034]    a database module including:  
           [0035]    at least one records database;  
           [0036]    a keywords database;  
           [0037]    a UAD database, and  
           [0038]    a UAD updating routine for updating UAD; and  
           [0039]    a ranking protocol including RSD and UAD for ranking retrieved records from the records database.  
           [0040]    The present invention provides a search engine component including:  
           [0041]    a packet processing module that converts each packet to at least one transaction;  
           [0042]    a ranking protocol including RSD and UAD for ranking retrieved records from the records database;  
           [0043]    a database module including  
           [0044]    at least one records database,  
           [0045]    a keyword database,  
           [0046]    a UAD database, and  
           [0047]    a UAD database updating routine for updating the UAD database a transaction processing module for processing transactions including:  
           [0048]    a new query processing routine for retrieving records satisfying the query and ranking the retrieved records according to the ranking protocol;  
           [0049]    a selected record processing routine including a first UAD activity value and a count incrementer,  
           [0050]    a record range processing routine including a second UAD activity value and the count incrementer for identifying retrieved records viewed by a user;  
           [0051]    a terminal activity processing routine including a third UAD activity value and the count incrementer.  
           [0052]    The search engine components of this invention can further include a query refinement module for identifying clustering of records based on UAD and RSD so that user queries can have improved relevance and context.  
           [0053]    The search engine components can be combined with other search engine components such as a user interface, RSD processing routines, communication and I/O routines, and like to produce complete search engines of the present invention.  
           [0054]    The present invention provides a database module including at least one database including records, keywords, UAD, and a UAD updating routine for updating UAD.  
           [0055]    The present invention includes a ranking protocol including RSD and UAD for ranking retrieved records from the records database.  
         METHODS AND PROCESSES  
         [0056]    The present invention also provides a method implemented in a search engine of the present invention stored in a DPU for capturing UAD involving receiving a user response, processing the user response, computing response UAD according to an user activity protocol if the response is not a new query, and updating UAD in appropriate databases in the engine.  
           [0057]    The present invention also provides a method implemented in a search engine of the present invention stored in a DPU for preprocessing a user query involving receiving a new query, determining query ambiguities, presenting to the user ambiguity resolving selections, and processing a user selected ambiguity resolving selection.  
           [0058]    The present invention also provides a method implemented in a search engine of the present invention stored in a DPU for updating a ranking protocol involving updating UAD based terms in the protocol.  
           [0059]    The present invention provides a method implemented in a search engine of the present invention stored in a DPU for searching involving receiving a user response and processing the user response. If the response is a new query, then records satisfying the query are assembled into a list. The assembled list is then ranked according to a ranking protocol comprising RSD and UAD terms and the ranked list is presented to the user. If the response is not a new query response or a termination response, then response UAD are computed according to a user activity protocol and corresponding UAD are updated in appropriate databases. The updated UAD is then used to refine the ranking protocol. The method is continued until the user session is completed. The method may also include the query preprocessing method described above. 
       
    
    
     DESCRIPTION OF THE DRAWINGS  
       [0060]    The invention can be better understood with reference to the following detailed description together with the appended illustrative drawings in which like elements are numbered the same:  
         [0061]    [0061]FIG. 1 is a block diagram of a main module of a search engine of the present invention;  
         [0062]    [0062]FIG. 2 is a block diagram of a new query module of a search engine of the present invention;  
         [0063]    [0063]FIG. 3 is a block diagram of a another page module of a search engine of the present invention;  
         [0064]    [0064]FIG. 4 is a block diagram of a record selected module of a search engine of the present invention;  
         [0065]    [0065]FIG. 5 is a block diagram of a housing-keeping module of a search engine of the present invention;  
         [0066]    [0066]FIG. 6 is a block diagram of a update UAD module of a search engine of the present invention;  
         [0067]    [0067]FIG. 7 is a block diagram of a query formulation module of a search engine of the present invention;  
         [0068]    [0068]FIG. 8 is a block diagram of a query results module of a search engine of the present invention;  
         [0069]    [0069]FIG. 9 is a block diagram of a second house-keeping module of a search engine of the present invention;  
         [0070]    [0070]FIG. 10 is a block diagram of a terminal data module of a search engine of the present invention;  
         [0071]    [0071]FIG. 11 is a block diagram of a link-list based database of a search engine of the present invention;  
         [0072]    [0072]FIG. 12 is a diagram of a record breakdown by user activity of a search engine of the present invention;  
         [0073]    [0073]FIG. 13 is a block diagram of modules of a search engine of the present invention;  
         [0074]    [0074]FIG. 14 is a block diagram of an preferred relational database containing UAD of a search engine of the present invention; and  
         [0075]    [0075]FIG. 15 is a block diagram of another preferred relational database containing UAD of a search engine of the present invention. 
     
    
     DETAILED DESCRIPTION OF THE INVENTION  
       [0076]    The inventor has found that a superior ranking protocol for database search hit list display organization can be constructed by incorporating user activity data. UAD includes at least selection activity data, but can also include time spent at selected record, record specific activity data and hypertext link activity data. UAD can either be directly used to augment the ranking protocol sequentially or UAD data can be accumulated in a database and used to continually refine ranking data. General—useful—1. more intelligent and discriminating in hits selection and display—2 becomes context sensitive for the purpose to query refinement (e.g. Beatles insect v. music; “Houston Rockets” basketball v. space)—3 prior art AI in engine modified by a module coupling user activity data a dynamic AI routine for aiding users to efficiency and effectively find applicable data related to therein query.  
         [0077]    Search Engines in General  
         [0078]    The term “search engine” is often used generically to describe both true search engines and directories. They are not the same. Additional details about search engines are described in U.S. Pat. Nos.: 5,890,152, 5,873,076, 5,870,740, 5,864,863, 5,864,846, 5,864,845, 5,819,255, and 5,802,524, incorporated herein by reference. The difference is how listings are compiled.  
         [0079]    Search Engines:  
         [0080]    Search engines, such as HotBot, create their listings automatically. Search engines crawl the web, then people search through what they have found. If you change your web pages, search engines eventually find these changes, and that can affect how you are listed. Page titles, body copy and other elements all play a role.  
         [0081]    Directories:  
         [0082]    A directory such as Yahoo depends on humans for its listings. You submit a short description to the directory for your entire site, or editors write one for sites they review. A search looks for matches only in the descriptions submitted. Changing your web pages has no effect on your listing. Things that are useful for improving a listing with a search engine have nothing to do with improving a listing in a directory. The only exception is that a good site, with good content, might be more likely to get reviewed than a poor site.  
         [0083]    Hybrid Search Engines:  
         [0084]    Some search engines maintain an associated directory. Being included in a search engine&#39;s directory is usually a combination of luck and quality. Sometimes you can “submit” your site for review, but there is no guarantee that it will be included. Reviewers often keep an eye on sites submitted to announcement places, then choose to add those that look appealing.  
         [0085]    Search Engines of the Present Invention  
         [0086]    Broadly, the present invention relates to a search engine or search engine add-on which after query capture, list acquisition, ranking and displaying, monitors, captures and saves user activity data involving the displayed list. The UAD can involve terminal activity (printing, e-mailing, bookmarking, copying/cutting/pasting, etc., temporal data associated with a listed record selection, and/or activity in a records after selection monitored by some type of an agent. For intranets and internets, the search engines of the present invention can use agents to monitor user activity once the user has jumped to a given site or URL. Each activity will have an associated value so that the UAD value and count can be updated for each record involved in a user activity. It should be recognized that the UAD is user independent and is accumulated upon any query activity involving the engine. Moreover, the engine could be designed to be user specific by including a user tag in each database entry, but this is not preferred because the database would become large rapidly. However, for user limited sites, such user specific UAD profiles may be extremely valuable.  
         [0087]    The search engines of the present invention also allow the engine query processing and listing ranking routines to learn as UAD is accumulated. Thus, the engine can learn better and more effective ways to response to a user query. Because most user queries are simple single word query, vast amount of data is likely to be retrieved, while the user may be interested only in a narrow subset of records that contain the single keyword. UAD can be used to break vast amount of data into focused classes or clusters of data. This classification can then be presented to the user prior to record retrieval to improve context and relevancy of the data eventually presented to the user. Additionally, the engine can learn to better rank retrieved records without going to the user. The ranking protocol withe usage can continually or periodically evolve over time.  
         [0088]    Search engine incorporating the UAD in its ranking protocols which can be toggled on or off without effecting the capturing of UAD for refining record UAD value and count.  
         [0089]    The present invention can further include routines for disabling UAD updating of keywords with “stable” ranking profile (i.e., the profile is not changing significantly with count) and to disable UAD updating for particular users and to disable UAD periodically or completely.  
         [0090]    The UAD data can then be used to construct a new ranking protocol which evolves as additional UAD data is captured, stored and analyzed. Thus, if a traditional ranking protocols relies solely on record specific data (RSD), then new ranking protocols of the present invention will comprise RSD and UAD to better service the user.  
         [0091]    UAD Storage  
         [0092]    The UAD data can be stored in a number of different locations so that the ranking protocol can be continuously updated and refined to better navigate user through a database. The UAD is preferably stored in the keyword indexed database and/or database table as a UAD value and count associated with each entry where each entry comprises a keyword keyed to a record in the record database, i. e., there is at least one entry for every keyword that appears in every record, but each entry is unique (a given keyword pointing to a given record). Each time a given keyword appears in a query and a given record is acted upon by the user, the UAD value and count associated with the entry corresponding to that keyword and record are updated by simple adding the current value and count with the query generated value and count.  
         [0093]    Alternatively and additionally, the UAD data can be stored in each record as a UAD weight and count. Thus, each time a user selects a given record, the activity generated UAD value is added to the current record UAD value and the record UAD count is incremented; again, the generated value and count can be an integer or a real number.  
         [0094]    Preferably, the UAD is stored in both the keyword database and the record database so that the engine will have keyword UAD as well and record UAD which can be used for query preprocessing and records database classifications.  
         [0095]    UAD Protocols—Value Assigments  
         [0096]    If the user query contains one or more terms and one or more operators, then the database can use any well-known process for updating UAD based on the relationship between the keywords and operators. The preferred method involves updating UAD equally for all keyword in an “AND” association in a user query, updating UAD individually for keywords in an “OR” association in a user query, (i.e., if keyword exists in record, then update; otherwise do not update) and updating UAD for all other queries which are combinations of keywords and boolean operators, as a decomposition in terms of “AND” and/or “OR” operations and/or additions or subtractions of lists generated by the “AND” and/or “OR” operations as is well known in the art.  
         [0097]    One preferred process for a assigning query specific UAD value uses the following numerical assignments for each user activity:  
                                       selected   +1       viewed and not selected   −1       not view and not selected   0       immediate return (return within j seconds)   −2       short time return (return between j seconds and k seconds)   +1       long time return (return between k seconds and l seconds)   +2       infinite time return (greater than 1 seconds)   0       printing a page list   +0.2       bookmarking   +3       e-mailing   +0.3       copy/cut/paste   +0.2                  
 
         [0098]    where j, k and l are counting or real numbers. Preferred values for j, k and l are: j about 30 seconds; k between about 30 seconds and about 180 seconds; l between about 180 seconds and about 1800 seconds, and greater than 1800 seconds the user is out to lunch. Thus, the UAD value associated with each keyword database entry can be updated by adding a query generated value to the current UAD value. The UAD counter is simply the number of times a user activity has involved the record. Although linear incrementation is preferred, the counter can be fractionally incremented with the fractional amount dependent on the type of user activity involved. Of course, any assignment protocol can be used as long as it discriminates based on user activity.  
         [0099]    Query Ambiguities and Query Preprocessing  
         [0100]    The present invention can also include query preprocessing routines that can be AI routines that analyze queries for ambiguities to partially alleviate data overload and to improve context and relevance of retrieved records. When a query is received, the routines will analyze the keywords that make up the query. Each keyword can appear n number of times in the keyword database, one entry for each appearance in a record. Each entry will have a UAD value and count. If the top ten entries having the highest values and/or counts can be selected, then the preprocessing routines jumps to the records and determine record-based contextual data by analyzing additional keyword frequencies or the like or the routine can find the UAD values and counts for each additional keyword that appears in the record and sorts the additional record keywords with respect to their weights and/or counts. The routine then causing the top m secondary or additional keywords to be presented to the user so that the scope of search can be modified to better suit the user.  
         [0101]    Alternatively, the preprocessing routine can analyze all of the retrieved keyword entries by retrieving all or a selected number of additional keywords in each retrieved record and applying a similarity classification protocol to cluster the records contextually, e.g., sports, entertainment, parks, roads, building, industries, housing, etc. and present the classification to the user.  
         [0102]    As UAD is accumulated and saved in the databases, where UAD include the queries as wells as all activity associated with query processing and hit displaying, the search engine or administrators of the search engine and analyze the UAD for the purpose of learning user selection patterns. These information can be used to better construct database, to better index data, to better perform house keeping functions, to stream line search strategies, to archive data of little or no relevance (not selected often) to reduce resource requirements or for any other purpose to which such information can be put.  
         [0103]    Ranking Protocol Refinement  
         [0104]    The search engines of the present invention also relates to methods for updating the ranking protocol used to rank records returned from the records database either continuously or periodically based on UAD. Additionally, the search engines of the present invention can present to user ranked data using different ranking protocols so that the routines can learn or construct preferred ranking protocols. Of course, if the search engine is using a particular form of a ranking protocol, the routine can learn directly from the accumulated UAD weights and counts.  
         [0105]    Ranking Protocols  
         [0106]    The present invention also relates to a ranking protocol including record specific data and user activity data. One embodiment of a ranking protocol is given by Formula (1):  
             r   =         ∑   i            α   i          w   i   pi         +       ∑     i   ,     i   ′                β     i   ,     i   ′         ·     (       w   i     ·     w     i   ′         )         -       ∑   j            α   j          w   j   pj         +       ∑     i   ,     j   ′                β     j   ,     j   ′         ·     (       w   j     ·     w     j   ′         )         +       ∑     i   ,   j              γ     i   ,   j       ·     (       w   i     ·     w   j       )                   (   1   )                               
 
         [0107]    where r is a ranking factor, α i  is a weight factor representing an amount of weight to be given to w i  which is a value of an i th  record specific criterion, p i  is a power factor of the i th  record specific criterion and i is an integer ranging over the record specific criteria; where α j  is a weight factor representing an amount of weight to be given to w j  which is a value of an j th  user activity criterion, p j  is a power factor of the j th  user activity criterion and j is an integer ranging over the user activity criteria and where β i,i′  is a weight factor for a product w i  time w i′ , β j,j′  is a weight factor for a product w j  time w j′ , and γ i,j   is a weight factor for a cross-product w i  times w j .  
         [0108]    Of course, it should be obvious to one of ordinary skill in the art that higher order products and cross-products can be included as well. It should be recognized that the w i &#39;s and w j &#39;s will vary over time. The w i &#39;s vary as new records are added while the w j &#39;s vary with UAD accumulation.  
         [0109]    One preferred ranking protocol of Formula (1) is given by Formula (2):  
             r   =         ∑   i            α   i          w   i   pi         +       ∑   j            α   j          w   j   pj                   (   2   )                               
 
         [0110]    where r, α i , w i , p i , i, α j , w j , p j  and j are as previously defined.  
         [0111]    Another preferred ranking protocol of Formula (1) is given by Formula (3):  
             r   =         ∑   i            α   i          w   i   pi         +       ∑   j            α   j          w   j                   (   3   )                               
 
         [0112]    where r, α i , w i , p i , i, α j , w j , and j are as previously defined.  
         [0113]    Another preferred ranking protocol of Formula (1) is given by Formula (4):  
             r   =         ∑   i            α   i          w   i         +       ∑   j            α   j          w   j   pj                   (   4   )                               
 
         [0114]    where r, α i , w i , i, α j , w j , p j  and j are as previously defined.  
         [0115]    Another preferred ranking protocol of Formula (1) is given by Formula (5):  
             r   =         ∑   i            α   i          w   i         +       ∑   j            α   j          w   j                   (   5   )                               
 
         [0116]    where r, α i , w i , i, α j , w j , and j are as previously defined.  
         [0117]    Another preferred ranking protocol of Formula (1) is given by Formula (6):  
               r   =       (       ∑   i            α   i          w   i   pi         )     +     β   ×     (       ∑   j          w   j   pj       )                   where                 r         ,     α   i     ,     w   i     ,     p   i     ,   i   ,     w   j     ,       p   j                   and                 j             (   6   )                               
 
         [0118]    are as previously defined and β is a weight factor for the entire indicated sum.  
         [0119]    Another preferred ranking protocol of Formula (1) is given by Formula (7):  
             r   =       β   ×     (       ∑   i          w   i   pi       )       +     (       ∑   j            α   j          w   j   pj         )               (   7   )                               
 
         [0120]    where r, w i , p i , i, α j , w j , j, α and β are as previously defined.  
         [0121]    Another preferred ranking protocol of Formula (1) is given by Formula (8):  
             r   =       α   ×     (       ∑   i          w   i       )       +     β   ×     (       ∑   j          w   j       )                 (   8   )                               
 
         [0122]    where r, w i , i, w j , p j , j, α and β are as previously defined. α and β have values that range between 0.0 and 1.0 and their sum is 1.0 so that the rank r will represent either a pure UAD ranking, a pure record selection ranking or a mixed ranking with a different blend of record selection data and UAD.  
         [0123]    Of course, ranking protocols can be generated that represent mixtures and combination of the rankings of formulas described above.  
         [0124]    UAD criteria include, without limitation, e-mailing, printing, bookmarking, marking, copying/cutting/pasting, selecting coupled with duration of selection, viewing, but not selected, or the like.  
         [0125]    RSD criteria include, without limitation, word frequency, word location, word proximity for multiple keyword queries, and all other record or site specific criterion identified in the US Patents cited and incorporated by reference herein and any other record or site specific data.  
         [0126]    Detailed Description of Conceptual Flowcharts  
         [0127]    Referring now to FIG. 1, a flowchart generally  100  is a server module called “Main”, which describes the basic steps for a program implemented in a digital processing unit for the purpose of receiving a packet that is sent to the search engine server for processing. This is an event driven module,  102  waits for an event to happen, i.e. a packet to show up in the queue, before this module is executed. The arrival of a new packet will create an event that will trigger this module. The first step in the execution is  104  where the module tries to determine, if the incoming packet is a “new query” or not. “New query” mean that the user has supplied one or more keywords and/or one or more Boolean operators, and the user wants to search for records that contain these keywords. If the determination is true, then the program proceeds along a YES branch to  106 , which is a call to an external module to process a new query. If the determination is false, then the routine proceeds along a NO branch to  108 .  108  is a step in which the module tries to determine if the incoming packet is “the user has selected a record” or not. This means the user has selected a record from the displayed list, and the user wants to view this record. Thus, this data needs to be logged in the database. If the determination is true, then the program proceeds along a YES branch to  110  which is a call to an external module to process the fact that the user has selected a record. If the determination is false, then the routine proceeds along a NO branch to  112 .  112  is a step in which the module tries to determine if the incoming packet is the “user has requested another page” or not. This means the user wants to see a different set of records. If the determination is true, then the program proceeds along a YES branch to  114  which is a call to an external module to process the fact that the user has requested another page. If the determination is false, then the routine proceeds along a NO branch to  116 .  116  is a step in which the module tries to determine if the incoming packet is reporting the user terminal activity data or not. This means this packet contains the data on the actions the user had performed on the client interface, i.e. print, bookmark, etc. If the determination is true, then the program proceeds along a YES branch to  118  which is a call to an external module to process the “terminal data”. If the determination is false, then the routine proceeds along a NO branch back to  102 , where the module will either wait or process the next packet in the queue.  
         [0128]    Referring now to FIG. 2, a flowchart generally  200  is a server module called “New Query”, which describes the basic steps for a program implemented in a digital processing unit for the purpose of processing a search request that is sent to the search engine. This is an event driven module,  202  waits for an event to happen, i.e. a packet to show up in the queue, before this module is executed. The arrival of a new packet will create an event that will trigger this module. The first step in the execution is  204  where the module will parse the keyword(s) and the Boolean operators (if any). Once the keyword(s) have been parsed, the next step  206  is to search the keyword index for the existence of the search keyword(s) so that the records that contain the keyword(s) can be determined. Step  208  will apply set theory operations on the keyword links to determine the records that have to be retrieved.  
         [0129]    Once it has been determined which records have to be retrieved, step  210  will issue a query to retrieve those records from the records database. Before the retrieved records can be sent to the user, the records have to be ranked, step  212  will rank the retrieved records based on UAD and other ranking protocols. Step  214  will open a new transaction for this user, the transaction will be timestamped, user address and the keyword(s) will be saved. Then the first set of records—a page—will be sent to the user.  
         [0130]    Referring now to FIG. 3, a flowchart generally  300  is a server module called “Record Selected”, which describes the basic steps for a program implemented in a digital processing unit for the purpose of processing the event that a certain user has selected a certain record from a displayed list of records. This is an event driven module,  302  waits for an event to happen, i.e. a packet to show up in the queue, before this module is executed.  
         [0131]    The arrival of a new packet will create an event that will trigger this module. The first step in the execution is  304  where the transaction is timestamped, so that subsequent time based calculations can be performed. In the next step  306  the module will try to determine if there is an open transaction that matches with this transaction. This is done in an effort to calculate the elapsed time, so that a user activity can be assigned a numeric value. If the determination is true, then the program proceeds along a YES branch to  308 . In step  308  the elapsed time between transactions is calculated. Contingent upon the elapsed time a weight is calculated.  
         [0132]    In step  310  the keyword-record count is incremented, and the keyword-record weight is updated. After step  310 , step  302  is performed.  
         [0133]    For step  306 , the determination is false, then the routine proceeds along a NO branch to  312 . In step  312  a transaction is opened, pertinent data such as user address, search keywords, records shown, etc. are stored, so it would be easier to perform subsequent calculations. After step  312  step  302  is performed.  
         [0134]    Referring now to FIG. 4, a flowchart generally  400  is a server module called “Another Page”, which describes the basic steps for a program implemented in a digital processing unit for the purpose of processing the event that a certain user has requested another set of records—page—of records. This is an event driven module,  402  waits for an event to happen, i.e. a packet to show up in the queue, before this module is executed. The arrival of a new packet will create an event that will trigger this module. The first step in the execution is  404  where the transaction is timestamped, so that subsequent time based calculations can be performed. In the next step  406  the module will try to determine the records that were displayed but not selected by the user, so that these records can be assigned a negative weight (because the user ignored these records which implies the records were not relevant), the transaction count will be incremented. In the next step  408  the module will try to determine the records—page—that are requested by the user. A transaction is opened for this activity, which will comprise of user address, keyword(s) and it is timestamped. After step  408  step  402  is performed.  
         [0135]    Referring now to FIG. 5, a flowchart generally  500  is a server module called “House-keeping”, which describes the basic steps for a program implemented in a digital processing unit for the purpose of processing the event that system has decided to do house-keeping, i.e. close transactions that have expired (been open for too long). This is an event driven module,  502  waits for an event to happen, i.e. a packet to show up in the queue, before this module is executed. The arrival of a new packet will create an event that will trigger this module. The first step in the execution is  504  where an open transaction is searched. In the next step  506  the module will try to determine if any open transaction was found or not. If the determination is false, then the program proceeds along a NO branch to  502  where it will process a new request. If the determination is true, then the program proceeds along a YES branch to step  508  where it will try to determine if the transaction has been open for longer than a predetermined period. If the determination is true, then the program proceeds along a YES branch to step  510  where it will close the open transaction, and update the UAD. Next step will be to transfer control to step  504 . If for step  508  the determination is false, then the program proceeds along a NO branch to step  512  where it will try to determine if all of the transactions have been processed. If the determination is true, then the program proceeds along a YES branch to step  502  where it will process a new packet. If the determination is false, then the program proceeds along a NO branch to step  504  where it will search for another open transaction.  
         [0136]    Referring now to FIG. 6, a flowchart generally  600  is a server module called “Update UAD”, which describes the basic steps for a program implemented in a digital processing unit for the purpose of updating the weights and counts based on UAD. This is an event driven module,  602  waits for an event to happen, i.e. a packet to show up in the queue, before this module is executed. The arrival of a new packet will create an event that will trigger this module. The first step in the execution is  604  where the module will try to determine if this packet is terminal data or not. If the determination is true, then the program proceeds along a YES branch to step  606  where an external routine “Terminal Data” is called to process this packet. If the determination is false, i.e. it is not terminal data, then the program proceeds along a NO branch to  608  where it will try to determine if this record was displayed or not. If the determination is false, i.e. the record was not displayed, then the program proceeds along a NO branch to  602  where it will process a new packet. If the determination is true, then the program proceeds along a YES branch to step  612  where it will try to determine if the record was selected by the user or not. If the determination is false, i.e. the record was not selected, then the program proceeds along a NO branch to  614  where it will update the UAD, a small negative value will be assigned to the weight, and the counter will be incremented. The next step will be  602  where it will process a new packet. At step  612  if the determination is true, then the program proceeds along a YES branch to step  616  where it will try to determine if the user viewed the packet for a short duration or not. If the determination is true, then the program proceeds along a YES branch to  618  where it will update the UAD, a large negative value will be assigned to the weight, and the counter will be incremented. The next step will be  602  where it will process a new packet. At step  616  if the determination is false, then the program proceeds along a NO branch to  620 . At step  620  the module will try determine if there were any subsequent user activity. If the determination is false, i.e. there was no subsequent user activity, then the program proceeds along a NO branch to  622  where it will update the UAD, a large positive value will be assigned to the weight, and the counter will be incremented. The next step will be  602  where it will process a new packet. At step  620  if the determination is true, then the program proceeds along a YES branch to step  624  where a small positive value will be assigned to the weight, and the counter will be incremented. The next step will be  602  where it will process a new packet.  
         [0137]    Referring now to FIG. 7, a flowchart generally  700  is a user module called “Query Formulation”, which describes the basic steps for a program implemented in a digital processing unit for the purpose of allowing an end user to create a query based on keyword(s) and/or keyword(s) and Boolean operator(s). In step  702  a user enters the desired query and submits it. In step  704  the user-interface supplements the query with additional data such as the user address (user address can be based on IP address, or CPU serial number, or a, combination of the two, etc.). Once the query packet has been sent to the server, the user-interface will stop  706  and wait for a response from the server.  
         [0138]    Referring now to FIG. 8, a flowchart generally  800  is a user module called “Query Result”, which describes the basic steps for a program implemented in a digital processing unit for the purpose of displaying the result of a user query and processing user activity. This is an event driven module, step  802  waits for a response back from the server, step  804  waits for a user event to happen, i.e. the user to perform some activity, before this module is executed. Any user activity will trigger this module. The first step in the execution is  806  where the module will try to determine if the user has modified the keywords or not, i.e. the user has changed his mind about the query keywords. If the determination is true, then the program proceeds along a YES branch to step  808  where an external routine “Send Query to Server” is called. If the determination is false, i.e. the user did not execute a new query, then the program proceeds along a NO branch to  810  where it will try to determine if the user selected another page or not. If the determination is true, then the program proceeds along a YES branch to step  812  where it will send a “Another Page” request and user data to the server. If the determination is false, i.e. another page was not requested, then the program proceeds along a NO branch to  814  where it will determine if the user selected a record or not. If the determination is true, then the program proceeds along a YES branch to step  816  where it will send the selected record and user data to the server. In step  818 , a call to an external routine will be performed where the selected record is displayed. If the determination is false, i.e. the user did not select a record, then the program proceeds along a NO branch to  820  where it will determine if any terminal activity (print, cut/copy/paste, bookmark, etc.) was performed. At step  820  if the determination is true, then the program proceeds along a YES branch to step  822  where it will capture the user terminal activity. If the determination is false, i.e. the user did not perform any terminal activity, then the program proceeds along a NO branch to  802  where it will process a new user activity.  
         [0139]    Referring now to FIG. 9, a flowchart generally  900  is a user module called “House Keeping”, which describes the basic steps for a program implemented in a digital processing unit for the purpose of processing data that has been captured by a cookie. Typically this will be the user terminal activity data. This is an event driven module, step  902  waits for an event to happen, i.e. a timer or when the CPU is idle, before this module is executed. The first step in the execution is  904  where the module will open a cookie. In step  906  the module will try to determine if any terminal data was stored in the cookie or not. If the determination is true, i.e. there was terminal data was stored in the cookie, then the program proceeds along a YES branch to step  908  where the necessary data is sent to the server. In step  910  the terminal data is purged from the cookie since it has been already sent to the server. Next step  902  will be performed. If for step  906  the determination is false, i.e. there was no terminal data in the cookie, then the program proceeds along a NO branch to  902  where it will process a another house-keeping event.  
         [0140]    Referring now to FIG. 10, a flowchart generally  1000  is a system module called “Terminal Data”, which describes the basic steps for a program implemented in a digital processing unit for the purpose of processing terminal activity data. This is an event driven module, step  1002  waits for an event to happen, before this module is executed. The first step in the execution is  1004  where the module will try to determine if the user printed a record or the page or not. If the determination is true, i.e. the user did print a record or the page, then the program proceeds along a YES branch to step  1006 , where for all of the records on the page a small positive weight is added to the keyword-record index and the activity counter is incremented. Next step  1002  is processed. If for step  1004  the determination is false, i.e. the user did not print a record or the page, then the program proceeds along a NO branch to  1008  where it will try to determine if the user copied a record or not. If the determination is true, i.e. the user did copy at least one record, then the program proceeds along a YES branch to step  1010 , where a small positive weight is added to the keyword-record index and the activity counter is incremented. Next step  1002  is processed. If for step  1008  the determination is false, i.e. the user did not copy a record, then the program proceeds along a NO branch to  1012  where it will try to determine if the user bookmarked a record or not. If the determination is true, i.e. the user did bookmark a record, then the program proceeds along a YES branch to step  1014 , where a large positive weight is added to the keyword-record index and the activity counter is incremented. Next step  1002  is processed. If for step  1012  the determination is false, i.e. the user did not bookmark a record, then the program proceeds along a NO branch to  1016  where it will try to determine if the user emailed this record or not. If the determination is true, i.e. the user did email a record, then the program proceeds along a YES branch to step  1018 , where a small positive weight is added to the keyword-record index and the activity counter is incremented. Next step  1002  is processed. If for step  1016  the determination is false, i.e. the user did not email a record, then the program proceeds along a NO branch to  1002  where it will process another “Terminal Data” event.  
         [0141]    Referring now to FIG. 11, generally  1100  which describes a link-list based database implemented in a digital processing unit for the purpose of storing and retrieving records.  1102  is a Record Database, it contains all of the necessary records.  1104  is a record in the Record Database  1102 , from  1106  to  1118  are records in the Record Database  1102 .  1130  is a Keyword Database,  1132  is a keyword in the Keyword Database  1130 .  1120  is a link-list that ties a keyword in the Keyword Database  1130  with the records that contain that keyword in the Record Database  1102 .  1138  is the link between the keyword  1132  and the link-list  1120 .  1128  (in the link-list  1120 ) is the index of a record in the Record Database  1102  that contains the keyword  1132 .  1126  is the weight associated with the keyword  1132  and the record  1116 .  1124  is the count associated with the keyword  1132  and the record  1116 .  1122  contains the address of the next link in the link-list. As new record(s) is added to the Record Database  1102 , new link will be added to the link-list  1120 , conversely as a record is deleted the link will be deleted. Of course, when new records are added, all associated database entries must be constructed.  
         [0142]    Referring now to FIG. 12, generally  1200  which describes a record breakdown by user query and subsequent activity.  1202  is the total number of records in the database.  1204  is the total number of records that satisfy the user query.  1206  is the total number of records that were displayed but not selected.  1208  is the total number of records that were selected and viewed for a longer duration.  1210  is the total number of records that were selected and viewed for a shorter duration.  
         [0143]    Referring now to FIG. 13, generally  1300  which describes the different modules of the system.  
         [0144]    [0144] 1304  is a Ranking Module, it ranks the records that satisfy a user query before the records are shown to the user. It is a server component.  
         [0145]    [0145] 1306  is a User Activity Capture Module, it captures the user activity and then sends it to the server. It is a client component.  
         [0146]    [0146] 1308  is a User Interface Module, it is the client interface, it allows the construction of query, capture of UAD and Terminal Data. It is a client component.  
         [0147]    [0147] 1310  is an Artificial Intelligence Module, it helps in determination of clusters, ranking of records, etc. It is a server component.  
         [0148]    [0148] 1312  is a Database Module, it contains the records, keywords, and UAD. It is a server component.  
         [0149]    [0149] 1314  is a Database Update Module, it updates the UAD. It is a server component.  
         [0150]    [0150] 1316  is a Query Execution Module, it processes and executes a query on the database.  
         [0151]    It is a server component.  
         [0152]    [0152] 1318  is a Communications and I/O Module, it allows the client and the server to communicate and read and write to the disk. It is both a client and a server component.  
         [0153]    Referring now to FIG. 14, generally  1400  which describes a relational database schema implemented in a digital processing unit for the purpose of storing and retrieving records.  1402  is a Records table, it contains all of the necessary records.  1404  is a unique record in the Records table  1402 ,  1406  is a unique key that will tie it to other table(s).  1410  is a Keywords table,  1412  is a keyword in the Keywords table  1410 , it is not unique.  1414  is a unique key that will tie it to other table(s). however, the combination of keyword  1412  and key  1414  is unique.  1416  will store the weight (negative or positive) for the keyword-record relation.  1418  will store the count, i.e. the number of times a user has performed some activity on this keyword-record relation.  1408  is a many-to-one relationship between the Keywords table  1410  and the Records table  1402 , i.e., many keywords can point to one record.  
         [0154]    Referring now to FIG. 15, generally  1500  which describes a relational database schema implemented in a digital processing unit for the purpose of storing and retrieving records.  1502  is a Records table, it contains all of the necessary records and UAD assigned to them.  1504  is a unique record in the Records table  1502 ,  1506  is a unique key that will tie it to other table(s).  1508  is the weight assigned to this record, i.e. how this record was ranked cumulatively.  1510  is the count assigned to this record, i.e. how many times user activity was recorded for this record.  1514  is a Keywords table,  1516  is a keyword in the Keywords table  1514 , it is not unique.  1518  is a unique key that will tie it to other table(s). However, the combination of keyword  1516  and key  1518  is unique.  1520  will store the weight (negative or positive) for the keyword-record relation.  1522  will store the count, i.e. the number of times a user has performed some activity on this keyword-record relation.  1512  is a many-to-one relationship between the Keywords table  1514  and the Records table  1502 , i.e. many keywords can point to one record.  
         [0155]    Although the invention has been disclosed with reference to its preferred embodiments, from reading this description those of skill in the art may appreciate changes and modification that may be made which do not depart from the scope and spirit of the invention as described above and claimed hereafter. All of these references are incorporated herein by reference.