Abstract:
A system and method provides recommendations of documents to a user of a document corpus. Document features are extracted and assigned weights, and a profile is likewise created for users. Documents are scored with respect to a given user based at least in part on the document features and the user&#39;s profile. The document scores may be adjusted to reflect organizational goals, such as promoting recommendation of newer documents. Based on the scores, recommendations are determined for a given user by identifying the top scores for that user and presented to the user in one of a variety of manners, such as within a web-based user interface, or via email. Interactions of the users with recommendations may be monitored and the recommendations updated accordingly.

Description:
BACKGROUND 
     1. Field of Art 
       [0001]    This invention relates generally to computer-implemented knowledge management systems and more specifically to computer systems that recommend to users of documents in document corpora. 
       2. Description of the Related Art 
       [0002]    Current computing systems make available vast quantities of digital documents, such as articles, technical talks, Wiki pages, slide shows, and the like. The sheer quantity of available data can make it difficult for users to locate the documents that are most pertinent to their particular interests. Recommendation systems address this problem by presenting the users with a selected set of documents chosen based on some prior knowledge of the user&#39;s interests. 
         [0003]    However, conventional recommendation systems have a number of shortcomings. For example, many conventional systems rely on domain-specific knowledge, such as customer habits regarding the purchase of movies. This places a great burden on the creator of the system to discover such knowledge and to design a custom recommendation system based on that knowledge, and does not permit an administrator to define corpora (i.e., distinct sets of documents) in a straightforward manner. Other conventional systems, such as many of those oriented towards retail sales, use social networking techniques (e.g., collaborative filtering), which rely on data about the interactions of other users with the various documents to infer the documents in which a particular user would be interested. However, the effectiveness of this technique is a function of the amount of the data on the interactions of other users, and thus systems with a small corpus or few users may not be able to beneficially employ social networking techniques. 
       SUMMARY 
       [0004]    Disclosed is a system and method for providing recommendations of documents to a user of a document corpus—i.e., a particular collection of documents, such as those relating to technical talks, books on science, and the like. In some organizational environments, there can be a number of distinct corpora, and each is administrable by a corpus administrator. In one embodiment, the corpora are further grouped according to a domain to which they belong. The present invention is of particular applicability where the number of documents and users of a given corpus is sufficiently small to be managed by a corpus administrator, or where there are a number of distinct corpora with which users of a single organization interact differently. These are scenarios in which conventional recommendation systems have low utility. 
         [0005]    In one embodiment, document features are extracted and assigned weights, and a profile is likewise created for the various users. Then, the documents are scored with respect to a given user based at least in part on the document features and the user&#39;s profile. The document scores are adjusted based on organization-specific information to reflect organizational goals, such as promoting recommendation of newer documents. Based on the scores, recommendations are determined for a given user by identifying the top scores for that user and the recommendations presented to the user. In one embodiment, recommendations are provided within a web-based user interface; in another they are provided via email; in another they are provided as an RSS feed; in still another they are provided as gadgets or frames embedded within other applications. Interactions of the users with recommendations are monitored and the recommendations updated accordingly. 
         [0006]    In one embodiment, a computer-implemented method presents to a user selected portions of an organization&#39;s corpora, the corpora comprising documents, the method being carried out by a processor configured to determine a set of weighted terms for each of a plurality of the documents, to construct a user profile including user interest areas, to calculate a score for each of the plurality of the documents based on correlation between the weighted terms and the user profile, to adjust the calculated scores based at least in part on rules specified by the organization, and to present the adjusted and scored items to the user. 
         [0007]    The features and advantages described in the specification are not all inclusive and, in particular, many additional features and advantages will be apparent to one of ordinary skill in the art in view of the drawings, specification, and claims. Moreover, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. 
     
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         [0008]    These and other features of the invention&#39;s embodiments are more fully described below. Reference is made throughout the description to the accompanying drawings, in which: 
           [0009]      FIG. 1  is a high-level block diagram illustrating a recommendation system for providing recommendations as described herein. 
           [0010]      FIG. 2  illustrates in more detail the components of the recommendation logic processor  115  of  FIG. 1 . 
           [0011]      FIG. 3  is a flowchart illustrating the process of providing recommendations, according to one embodiment. 
           [0012]      FIG. 4  illustrates a user interface for displaying and interacting with recommendations. 
           [0013]      FIGS. 5A-D  illustrate user interfaces for administration of various aspects of the recommendation system  110  of  FIG. 1 . 
           [0014]      FIG. 6  illustrates a general purpose computer for use in implementing recommendation logic processor  115  of  FIG. 1 . 
       
    
    
     DETAILED DESCRIPTION 
     System Architecture 
       [0015]      FIG. 1  is a high-level block diagram illustrating a recommendation system  110  for providing the recommendations described herein. Also illustrated are a client computer system  120  used to interact with and/or receive recommendations from the recommendation system  110 , as well as a network  150  facilitating communications between the client  120  and the recommendation system  110 . 
         [0016]    The recommendation system  110  comprises a corpus definitions database  111 , which defines each corpus in the system. In one embodiment, a corpus has a name, a set of associated documents, and (optionally) a set of associated users. As used herein, a “document” is a digital representation of information. A word processing file is a common example, but documents include many other things as well, such as digital representations of calendared events (e.g. a talk scheduled for a particular place at a particular time). The associated documents need not be stored on the recommendation system  110  itself; rather, in one embodiment only identifiers (e.g., URLs, path and file names) of the documents themselves need be stored—the data for the documents can be stored on the recommendation system  110 , on systems available on a network (e.g.,  150 ) that is local to the recommendation system  110 , or on a remote system. In one embodiment, the associated users are represented by identifiers, such as operating system user IDs, of users interested in documents pertaining to that particular corpus. The documents for a given corpus need not be all of the same operating system file type, e.g. a text file or presentation file for a particular presentation application software, but rather can represent the conceptual category of the corpus. For example, in an exemplary embodiment one corpus is named “Technical Presentations,” has a set of 20 associated technical presentations in formats such as ADOBE PDF, Microsoft PowerPoint, word processing formats, event announcements and the like, and has two hundred associated users. 
         [0017]    In one embodiment, the corpora are further grouped into domains. For example, an organization administering the recommendation system  110  creates individual domains for each organization that wishes to obtain its own personalized access to the recommendation system  110 . The implementation for this embodiment is similar to that described above, with the addition of an association between a corpus and a domain. 
         [0018]    A document features repository  112  stores a set of features for each document of the various corpora defined by corpus definitions repository  111 . Features of a document represent its concepts, and in one embodiment consist of words and multi-word phrases (“n-grams”). In one example, a document on fishing has associated with it in the features repository  112  the set of terms “salmon”, “fly”, “reel”, “rod”, and “fishing vessel”, with each having an associated value (also referred to as a “weight”) quantifying how relevant the term is to the document. Features of a document could be present in the document itself, could be derived from a user-specified label, or could represent a category to which the document was assigned (e.g. “technical presentations”), for example. In one embodiment, the terms are chosen from a discrete set of possible terms, such as a set of 50,000 terms known to be useful in characterizing a document for search and recommendation purposes. As with other data storage repositories described below, the document features repository  112  is implemented in a conventional manner, such as a table of a conventional relational database management system, a text file, or a specialized binary file. Other manners of implementing repository  112  will be known to one of skill in the art. 
         [0019]    A profile features repository  113  stores features, such as terms, associated with users. In one embodiment, each user has an associated profile, the profile storing terms chosen from the same set of possible terms as for the document features repository. The terms represent the interest areas of the user, each having an associated weight quantifying the relevance of the term to the user. As described further below, the terms and their weightings are derived from sources such as documents associated with the user, areas of interest explicitly entered by the user, and user interactions with recommended documents. 
         [0020]    A document scores repository  114  stores scores for the various documents identified in the corpus definitions repository  111 , each score quantifying the relevance of a given document to a given user. In one embodiment, the score is calculated based at least in part on a function of a profile for the user and the document features for the document. 
         [0021]    Recommendation logic processor  115 , as described further below, is a subsystem that determines which documents are most relevant to a given user, and provides a list of the recommended documents to the user. 
         [0022]    A corpora management interface  116  provides a user interface allowing administration of corpora. A root user interface allows a root administrator responsible for administration of the recommendation system  110  as a whole to perform tasks such as adding new domains, e.g. by specifying a new document type. A corpus administrator interface allows a corpus administrator to perform tasks such as adding new corpora (e.g., by specifying a new document type), specifying which documents should be included within the corpus, specifying when document features and scores should be calculated or recalculated, and the like. Such features are illustrated in more detail with respect to  FIGS. 5A-5C . 
         [0023]    A corpus recommendation user interface module  117  generates the user interface displaying and allowing interaction with the recommendations for a particular corpus. In one embodiment, the user interface is constructed using a browser-based scripting language such as JavaScript, which can be rendered within a conventional web browser, e.g. as a particular module added by a user to a web page. 
         [0024]      FIG. 2  illustrates in more detail the conceptual components of the recommendation logic processor  115  of  FIG. 1 . Referring now also to  FIG. 6 , in an exemplary embodiment recommendation logic processor  115  is, along with other aspects of system  110 , implemented by programming a general purpose computer  600 . Illustrated are a processor  602  coupled to a bus  604 . Also coupled to the bus  604  are a memory  606 , a storage device  608 , a keyboard  610 , a graphics adapter  612 , a pointing device  614 , and a network adapter  616 . A display  618  is coupled to the graphics adapter  612 . The processor  602  is in one embodiment any general-purpose processor such as an INTEL x86 compatible-CPU. The memory  606  can be firmware, read-only memory (ROM), non-volatile random access memory (NVRAM), and/or RAM, which holds instructions and data used by the processor  602 . The memory  606  may be divided into pages by an operating system of the computer  600 , each page having attributes such as whether the page is readable, writable, or executable (i.e. contains executable instructions), or whether it was loaded from a file on the storage device  608 . The storage device  608  is, in one embodiment, a hard disk drive but can also be any other device capable of storing data, such as a writeable compact disk (CD) or DVD, a solid-state memory device, or other form of computer-readable storage medium. The storage device  608  stores files and other data structures used by the computer  600 . 
         [0025]    Referring again back to  FIG. 2 , a feature extraction module  210  parses documents, assigning features with weights, or scores, according to a weighting algorithm. In an exemplary embodiment, a conventional term-frequency/inverse document frequency (“tf/idf”) weighting algorithm is used, in which each possible term (e.g., the 50,000 useful terms referenced above) is located in the document, and the term&#39;s calculated weight is proportional to the number of times the term appears in the document and inversely proportional to the frequency of the word in the corpus. Further detail on document weighting and scoring is found, for example, in commonly owned U.S. Pat. No. 7,383,258 to Georges Harik and Noam Shazeer, entitled “Method and Apparatus for Characterizing Documents Based on Clusters of Related Words.” 
         [0026]    In one embodiment, the features and weights extracted automatically by the weighting algorithm are supplemented by additional features and weights associated with the document, such as any tags that the user has associated with the document. The features and weights are then stored in the document features repository  112  in association with an identifier of the document from which they were extracted. 
         [0027]    A profile construction module  220  populates the profile features repository  113 . In one embodiment, the profile construction module  220  creates an initial profile for a given user based on available data sources. One data source is directory information available within the organization having the domain or corpus of which the user is a member, such as Lightweight Directory Access Protocol (LDAP) information stored on the organization&#39;s directory servers, e.g. personnel data available within a company tracking attributes such as age, sex, department, and the like. Another data source is a set of particular non-directory documents associated with the user and stored within the organization, such as a resume of the user or other document indicative of the user&#39;s interest areas. Terms are extracted from the document and weighted using the algorithms described above. 
         [0028]    Use of these data sources allows the organization to leverage existing information that it stores about the user to produce higher-quality profiles than are created for systems which lack such pre-existing data about the user. In another embodiment, the user explicitly indicates terms of interest, such as by specifying a set of keywords (e.g. “tennis”, “Victorian literature”, etc.). Such explicitly-indicated terms in one embodiment are then assigned a weight higher than the weight of any other non-explicitly-indicated terms, representing the high degree of utility of explicit interests. In one embodiment, the profile construction module  220  additionally updates a user&#39;s profile, e.g. based on interactions of the user with documents, such as viewing initially, viewing for some period of time, printing, saving, emailing, explicitly marking the document as favored or disfavored using a user interface, and the like. For example, if a user is provided with a set of recommended documents and views a document having given terms, the value of those terms within the user&#39;s profile within the profiles feature repository  113  can be increased by an appropriate amount. In one embodiment, the effect of an interaction on the value of terms within the profile may decrease over time as the interaction ages. In one embodiment, the particular interaction triggering the update of the profile term value leads to different profile update actions. In an example, viewing of the document leads to a lesser increase in the value than printing the document, an action that presumably indicates more serious interest on the part of the user than does viewing. As another example, marking a document as disfavored leads not merely to reducing the values in the user profile for the terms within the document, but also to removing that article, possibly permanently, from any recommendations later provided to that user. 
         [0029]    A document score calculator  230  calculates a score for a given document with respect to a given user based on a correlation between the feature weights generated by the feature extraction module  210  and the profiles generated by the profile construction module  220 . In one embodiment, the correlation algorithm is a conventional cosine similarity algorithm, which calculates the cosine of e.g. tf-idf vectors of terms for the user&#39;s profile and for the document being scored. In another embodiment, the document scores are not calculated independently of each other, but rather influence each other. In one example, a document scoring algorithm is designed to spread knowledge throughout the organization by recommending every document of the corpus to at least one user of the corpus. Such an approach is useful for avoiding institutional knowledge gaps that can come to exist for reasons such as employee attrition. This algorithm addresses an optimization problem in which the goal is to maximize the standard correlation measure matches between users and documents and to minimize the overlap (or maximize the completeness) of the coverage of all the documents in the corpus by the employees. The scores are calculated with respect to all of the users of the corpus at once through conjugate gradient, Monte Carlo, or other optimization techniques. In some embodiments, a number of algorithms are available, and the choice of which particular algorithm to use for a given corpus is made by the corpus administrator via the corpora management interface  116 . The document scores are then stored in the document scores repository  114  in association with an identifier of the user and the document to which they correspond. 
       Method of Operation 
       [0030]      FIG. 3  is a flowchart illustrating the process of providing recommendations, according to one embodiment. At step  310 , document features of documents in the corpus are weighted, such as by the feature extraction module  210 . As discussed above, this entails, for each document, assigning weights, or scores, to the document features according to a weighting algorithm. One conventional weighting algorithm is term-frequency/inverse document frequency (“tf/idf”). In one embodiment, the features and weights extracted automatically by the above algorithms are supplemented by additional features and weights associated with the document, such as user-defined tags. The features and weights are then stored in the document features repository  112 . 
         [0031]    At step  320 , which may be performed before, in parallel with, or after step  310 , an initial profile is created for a user, as described above with respect to the profile construction module of  FIG. 2 . 
         [0032]    At step  330 , documents from corpora  130  are scored by the document score calculator  230  as described above with respect to  FIG. 2 . In one embodiment the scoring is initiated manually, e.g. through a user interface provided by corpora management interface  116 ; in another embodiment scoring is initiated at scheduled intervals, such as through a Unix “cron” process or other form of scheduled task. 
         [0033]    At step  340 , the document scores are adjusted as desired based on the current context and the document features. A number of different adjustment rules may be used, and in one embodiment are specified by the corpus administrator via the corpora management interface  150 . For example, one adjustment rule biases the score in favor of more recent documents, e.g. by calculating an amount of time between a date of the document (e.g., a creation or modification date) and a set date, increasing the score as a function of the calculated amount of time if the document date is after the set date, and decreasing it otherwise. Adjustment of scores based on document recency can also be accomplished via exponential decay according to a specified document half-life. Another rule biases the score based on the document type or the document itself, e.g. specifying a multiplier value for the score of documents of type “tech talk”, or for a specified “tech talk” document deemed (e.g., by the corpus administrator) to be of particular interest. Still another rule increases the weight of documents that are specific to a user&#39;s organization (e.g., company) and increases the weight yet further for documents that are specific to the department or unit of the organization in which the user works. Such rules can also be used to limit the number of results, e.g. through a specified maximum number of results or through a specified minimum score (i.e., a threshold). 
         [0034]    At step  350 , recommendations for a particular user are determined by the recommendation provider module  240 , as described above. They are then provided to the user. In one example, the results are displayed within the user interface provided by the corpus recommendation UI, such as the corpus recommendation user interfaces discussed with respect to  FIG. 4 , below. In another example, the recommendations are emailed to the user. In still another example, the recommendations are provided as an RSS feed and displayed within an RSS viewer whenever a new recommendation is added to the list. 
         [0035]    At step  360 , the user&#39;s interactions with documents are monitored. As previously described, different interactions with a document could indicate an interest level of the user in the document, such as viewing, printing, emailing, saving, explicitly marking as favored or disfavored, and the like. 
         [0036]    At step  370 , if the user interactions monitored at step  360  result in a modification of the user&#39;s profile, then the recommendations for that user are likewise updated. 
         [0037]      FIG. 4  illustrates a user interface for displaying and interacting with recommendations. Displayed are user interfaces representing recommendations for four corpora,  401 A- 401 D. Each has a title  405 A and a set of recommended documents such as  410 A. Recommended document  410  has associated “thumbs up” and “thumbs down” icons which the user may select to indicate interest or lack of interest in the article, which are used to update the user profile as described above with respect to profile construction module  220 . Each also has an options bar, e.g.,  425 A, which lists various options associated with the corpus. A corpus recommendation user interface  401 A also provides options such as RSS icon  420 A, which causes changes to recommendations to be delivered to a news reader of the subscriber via the RSS protocol. 
         [0038]    A user interface such as that of  FIG. 4  displays all of the corpus recommendation user interfaces  401  associated with a user. Alternatively, individual corpus recommendation user interfaces  401  can be individually embedded within other user interfaces. For example, a web site could support the use of such corpus recommendation user interfaces  401  by allowing a user to select one or more corpus recommendation user interfaces of interest to be embedded in a user&#39;s personal home page, for example, and subsequent accesses of that home page by the user could fetch the user interface from the corpus recommendation user interface module  117  of the recommendation system  110 . 
         [0039]      FIGS. 5A-D  illustrate user interfaces for administration of various aspects of the recommendation system  110  of  FIG. 1 .  FIG. 5A  illustrates a user interface for a root administrator. User interface area  505  allows a root administrator using the interface to grant rights to the root domain to another user. User interface area  510  allows the root administrator to make a user an administrator of a given domain. That user will then have permissions to administer that domain as described in  FIG. 5B , below. User interface area  515  allows the root administrator to create a new domain, and individual corpora can then be associated with that domain, e.g. by an administrator for the domain. Finally, user interface area  520  allows the root administrator to see a list of all the domains that have been created for the recommendation system  110 . 
         [0040]      FIG. 5B  illustrates a user interface for an administrator of one of the domains. User interface area  530  allows the domain administrator to add a new corpus to the domain, optionally specifying both a full name of the domain (e.g., “Network Security Forum”) and a short name (e.g., “Net. SF”) for use in areas of user interfaces of the recommendation system in which compact names are useful. User interface area  535  allows the domain administrator to grant corpus administration privileges to another user of the system for one of the corpora in the domain—in the illustrated example, the corpus named “Jobs.” User interface area  540  allows the domain administrator to make another user of the system an administrator of the same domain. 
         [0041]      FIG. 5C  illustrates a user interface for a domain administrator for setting the default attributes for any corpus in that domain. An equivalent interface is used by an administrator of a specific corpus to define behavior of the recommendation and presentation of the documents in the corpus. User interface area  540  allows the corpus administrator to specify Javascript code to define the user interface of the corpus recommendation as desired. For example, the corpus administrator could write code to add menu items, links, etc. to the user interface of the corpus recommendation user interface, such as the link bar  425 A of  FIG. 4 . User interface area  555  allows the corpus administrator to specify Cascading Style Sheet (CSS) code to control visual aspects such as how the document link text is displayed (9 point Arial font in the illustrated example). User interface area  560  allows the corpus administrator to specify which scoring algorithms and score adjustment filters to employ when computing scores for documents in the corpus. User interface area  565  allows the corpus administrator to specify “stop words”, i.e., words that will be ignored when scoring the documents in the corpus. Finally, user interface area  570  allows the corpus administrator to specify attributes of the corpus, such as the algorithm used to weight document features (e.g., “KW” in this example, which refers to the tf-idf, or “keyword”, algorithm). 
         [0042]    It is appreciated that methods carrying out the above-described steps need not include the exact steps, formulas, or algorithms disclosed above, nor need they be in the same precise order. Rather, variations on the scope and functionality of the individual steps, and on the order thereof, are possible while still accomplishing the aims of the present invention. 
         [0043]    As used herein any reference to “one embodiment” or “an embodiment” means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment. 
         [0044]    As used herein, the terms “comprises,” “comprising,” “includes,” “including,” “has,” “having” or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Further, unless expressly stated to the contrary, “or” refers to an inclusive or and not to an exclusive or. For example, a condition A or B is satisfied by any one of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present). 
         [0045]    In addition, the words “a” or “an” are employed to describe elements and components of the invention. This is done merely for convenience and to give a general sense of the invention. This description should be read to include one or at least one and the singular also includes the plural unless it is obvious that it is meant otherwise. 
         [0046]    Certain aspects of the present invention include process steps and instructions described herein in the form of a method. It should be noted that the process steps and instructions of the present invention could be embodied in software, firmware or hardware, and when embodied in software, could be downloaded to reside on and be operated from different platforms used by real time network operating systems. 
         [0047]    The present invention also relates to a system for performing the operations herein. This system may be specially constructed for the required purposes, or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored on a computer readable medium that can be accessed by the computer. Such a computer program may be stored in a computer readable storage medium, such as, but is not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, application specific integrated circuits (ASICs), or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus. Furthermore, the computers referred to in the specification may include a single processor or may be architectures employing multiple processor designs for increased computing capability. 
         [0048]    Upon reading this disclosure, those of skill in the art will appreciate that still additional alternative structural and functional designs are possible. Thus, while particular embodiments and applications have been illustrated and described, it is to be understood that the present invention is not limited to the precise construction and components disclosed herein and that various modifications, changes and variations which will be apparent to those skilled in the art may be made in the arrangement, operation and details of the method and apparatus of the present invention disclosed herein without departing from the spirit and scope of the invention as defined in the appended claims.